source: sans/Dev/trunk/NCNR_User_Procedures/Reduction/SANS/EventModeProcessing.ipf @ 1241

Last change on this file since 1241 was 1241, checked in by srkline, 3 years ago

fix in VSANS transmission to name the panel when updating popup controls

addition to SANS event mode to speedup the non-XOP event file loader. old veriosn was 28x slower than the xop. this version is now only about 5x slower. Still not great, but tolerable if the XOP is not present (macOS notarization issues)

File size: 131.6 KB
Line 
1#pragma rtGlobals=1             // Use modern global access method.
2#pragma IgorVersion=6.22
3
4// vers 7.13e
5
6// TODO:
7//
8// -- search for TODO for unresolved issues not on this list
9//
10// -- add comments to the code as needed
11//
12// -- write the help file, and link the help buttons to the help docs
13//
14// -- examples?
15//
16// -- added a test function AutoFix_Rollover_Steps() that will search for steps (simple delta) that
17//    are +/- 0.1s away from the 6.7s missed rollover. This function looks for up, down, then re-calculates
18//    the derivative. ?? should this be set to work only between the cursors? Currently it does the whole set.
19//
20// -- added a test function PutCursorsAtBigStep(tol). that will look for a big "jump", larger than tol.
21//    find a way to work this into the panel, maybe add a button and a setVar to set the value
22//
23//
24// -- EC_FindStepButton_down (and up) now both do the SAME thing -- using the "PutCursorsAtBigStep" function
25//    as above. This may change in the future...
26//
27//
28// -- NEW 2016
29//
30// -- see the Procedure DecodeBinaryEvents(numToPrint) for a first implementation of a "viewer"
31//    for the binary events. Prints out the decoded 32-bit binary to the command window for inspection
32//    Could be modified as needed to flag events, see what bad events really look like, etc. (May 2016)
33
34//  -- see the proc DisplayForSlicing() that may aid in setting the bins properly by plotting the
35//     bins along with the differential collection rate
36//
37// -- ?? need way to get correspondence between .hst files and VAX files? Names are all different. See
38//    DateAndTime2HSTName() functions and similar @ bottom of this file
39//
40// x- add the XOP to the distribution package
41//
42// x- Need to make sure that the rescaledTime and the differentiated time graphs are
43//     being properly updated when the data is processed, modified, etc.
44//
45// -- I need better nomenclature other than "stream" for the "continuous" data set.
46//     It's all a stream, just sometimes it's not oscillatory
47//
48//
49// X- the slice display "fails" for data sets that have 3 or 4 slices, as the ModifyImage command
50//     interprets the data as being RGB - and so does nothing.
51//     need to find a way around this. This was fixed by displaying the data using the G=1 flag on AppendImage
52//     to prevent the "atuo-detection" of data as RGB
53//
54// -- Do something with the PP events. Currently, only the PP events that are XY (just the
55//    type 0 events (since I still need to find out what they realy mean)
56//
57// -- Add a switch to allow Sorting of the Stream data to remove the "time-reversed" data
58//     points. Maybe not kosher, but would clean things up.
59//
60// -- when a large stream file is split, and each segment is edited, it is re-loaded from a list and is currently
61//    NOT decimated, and there is no option. There could be an option for this, especially for these
62//    large files. The issue is that the data segments are concatentated (adjusting the time) and can
63//    be too large to handle (a 420 MB event file split and edited and re-loaded becomes a 1.6 GB Igor experiment!!)
64//
65//    -- so allow the option of either decimation of the segments as they are loaded, or create the option to
66//       bin on the fly so that the full concatentated set does not need to be stored. Users could be
67//       directed to determine the binning and manipulate the data of a sufficiently decimated data set
68//       first to set the binning, then only process the full set.
69//
70///////////////////////////////
71//
72// NOTE -- to be able to show the T0 and PP event locations/times, force the loader to use the Igor code rather than
73// the XOP. Then there will be waves generated with their locations: T0Time and T0EventNum, PPTime and PPEventNum. Edit these waves
74// and (1) delete the zero points at the end of the waves and (2) multiply the Time wave * 1e-7 to convert to seconds.
75// Then the waves can be plotted on top of the event data, so it can be seen where these events were identified.
76//
77///////////////   SWITCHES     /////////////////
78//
79// for the "File Too Big" limit:
80//      Variable/G root:Packages:NIST:Event:gEventFileTooLarge = 150            // 150 MB considered too large
81//
82// for the tolerance of "step" detection
83//      Variable/G root:Packages:NIST:Event:gStepTolerance = 5          // 5 = # of standard deviations from mean. See PutCursorsAtStep()
84//
85//
86///////// DONE //////////
87//
88// X- memory issues:
89//              -- in LoadEvents -- should I change the MAKE to:
90//                              /I/U is unsigned 32-bit integer (for the time)
91//                              /B/U is unsigned 8-bit integer (max val=255) for the x and y values
92//                      -- then how does this affect downstream processing - such as rescaledTime, differentiation, etc.
93//                      x- and can I re-write the XOP to create these types of data waves, and properly fill them...
94//
95//  **- any integer waves must be translated by Igor into FP to be able to be displayed or for any
96//    type of analysis. so it's largely a waste of time to use integers. so simply force the XOP to
97//    generate only SP waves. this will at least save some space.
98//
99//
100//
101// x- Is there any way to improve the speed of the loader? How could an XOP be structured
102//     for maximum flexibility? Leave the post processing to Igor, but how much for the XOP
103//     to do? And can it handle such large amounts of data to pass back and forth, or
104//     does it need to be written as an operation, rather than a function??? I'd really
105//     rather that Igor handles the memory management, not me, if I write the XOP.
106//
107// **- as of 11/27, the OSX version of the XOP event loader is about 35x faster for the load!
108//    and is taking approx 1.8s/28MB, or about 6.5s/100MB of file. quite reasonable now, and
109//    probably a bit faster yet on the PC.
110//
111//
112// X- fix the log/lin display - it's not working correctly
113//                      I could use ModifyImage and log = 0|1 keyword for the log Z display
114//                      rather than creating a duplicate wave of log(data)
115//                      -- it's in the Function sliceSelectEvent_Proc()
116//
117// X- add controls to show the bar graph
118// x- add popup for selecting the binning type
119// x- add ability to save the slices to RAW VAX files
120// X- add control to show the bin counts and bin end times
121// x- ADD buttons, switches, etc for the oscillatory mode - so that this can be accessed
122//
123// x- How are the headers filled for the VAX files from Teabag???
124// -- I currently read the events 2x. Once to count the events to make the waves the proper
125//     size, then a second time to actualy process the events. Would it be faster to insert points
126//     as needed, or to estimate the size, and make it too large, then trim at the end...
127// ((( NO -- I have no good way of getting a proper estimate of how many XY events there are for a file))
128//
129//
130//
131
132
133//
134// These are also defined in the TISANE procedure file. In both files they are declared
135// as Static, so they are local to each procedure
136//
137Static Constant ATXY = 0
138Static Constant ATXYM = 2
139Static Constant ATMIR = 1
140Static Constant ATMAR = 3
141
142Static Constant USECSPERTICK=0.1 // microseconds
143Static Constant TICKSPERUSEC=10
144Static Constant XBINS=128
145Static Constant YBINS=128
146//
147
148Static Constant MODE_STREAM = 0
149Static Constant MODE_OSCILL = 1
150Static Constant MODE_TISANE = 2
151Static Constant MODE_TOF = 3
152
153//Menu "Macros"
154//      "Split Large File",SplitBigFile()
155//      "Accumulate First Slice",AccumulateSlices(0)
156//      "Add Current Slice",AccumulateSlices(1)
157//      "Display Accumulated Slices",AccumulateSlices(2)       
158//End
159
160
161
162Proc Show_Event_Panel()
163        DoWindow/F EventModePanel
164        if(V_flag ==0)
165                Init_Event()
166                EventModePanel()
167        EndIf
168End
169
170
171Function Init_Event()
172
173        NewDataFolder/O/S root:Packages:NIST:Event
174
175        String/G        root:Packages:NIST:Event:gEvent_logfile
176        String/G        root:Packages:NIST:Event:gEventDisplayString="Details of the file load"
177       
178        Variable/G      root:Packages:NIST:Event:AIMTYPE_XY=0 // XY Event
179        Variable/G      root:Packages:NIST:Event:AIMTYPE_XYM=2 // XY Minor event
180        Variable/G      root:Packages:NIST:Event:AIMTYPE_MIR=1 // Minor rollover event
181        Variable/G      root:Packages:NIST:Event:AIMTYPE_MAR=3 // Major rollover event
182
183        Variable/G root:Packages:NIST:Event:gEvent_time_msw = 0
184        Variable/G root:Packages:NIST:Event:gEvent_time_lsw = 0
185        Variable/G root:Packages:NIST:Event:gEvent_t_longest = 0
186
187        Variable/G root:Packages:NIST:Event:gEvent_tsdisp //Displayed slice
188        Variable/G root:Packages:NIST:Event:gEvent_nslices = 10  //Number of time slices
189       
190        Variable/G root:Packages:NIST:Event:gEvent_logint = 1
191
192        Variable/G root:Packages:NIST:Event:gEvent_Mode = MODE_OSCILL                           // ==0 for "stream", ==1 for Oscillatory
193        Variable/G root:Packages:NIST:Event:gRemoveBadEvents = 1                // ==1 to remove "bad" events, ==0 to read "as-is"
194        Variable/G root:Packages:NIST:Event:gSortStreamEvents = 0               // ==1 to sort the event stream, a last resort for a stream of data
195       
196        Variable/G root:Packages:NIST:Event:gEvent_ForceTmaxBin=1               //==1 to enforce t_longest in user-defined custom bins
197
198        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
199       
200               
201        Make/D/O/N=(XBINS,YBINS,nslices) slicedData
202        Duplicate/O slicedData logslicedData
203        Duplicate/O slicedData dispsliceData
204
205
206// for decimation
207        Variable/G root:Packages:NIST:Event:gEventFileTooLarge = 2500           // 2500 MB considered too large
208        Variable/G root:Packages:NIST:Event:gDecimation = 100
209        Variable/G root:Packages:NIST:Event:gEvent_t_longest_decimated = 0
210        Variable/G root:Packages:NIST:Event:gEvent_t_segment_start = 0
211        Variable/G root:Packages:NIST:Event:gEvent_t_segment_end = 0
212
213// for large file splitting
214        String/G root:Packages:NIST:Event:gSplitFileList = ""           // a list of the file names as split
215       
216// for editing
217        Variable/G root:Packages:NIST:Event:gStepTolerance = 5          // 5 = # of standard deviations from mean. See PutCursorsAtStep()
218       
219        SetDataFolder root:
220End
221
222//
223// -- extra bits of buttons... not used
224//
225//      Button button9 title="Decimation",size={100,20},pos={490,400},proc=E_ShowDecimateButton
226//
227//      Button button11,pos={490,245},size={150,20},proc=LoadDecimateButtonProc,title="Load and Decimate"
228//      Button button12,pos={490,277},size={150,20},proc=ConcatenateButtonProc,title="Concatenate"
229//      Button button13,pos={490,305},size={150,20},proc=DisplayConcatenatedButtonProc,title="Display Concatenated"
230//     
231//      GroupBox group0 title="Manual Controls",size={185,112},pos={490,220}
232//
233//      NewPanel /W=(82,44,854,664)/N=EventModePanel/K=2
234//      DoWindow/C EventModePanel
235//      ModifyPanel fixedSize=1,noEdit =1
236Proc EventModePanel()
237        PauseUpdate; Silent 1           // building window...
238        NewPanel /W=(82,44,884,664)/N=EventModePanel/K=2
239        DoWindow/C EventModePanel
240//      ModifyPanel fixedSize=1,noEdit =1
241
242        SetDrawLayer UserBack
243        DrawText 479,345,"Stream Data"
244        DrawLine 563,338,775,338
245        DrawText 479,419,"Oscillatory Data"
246        DrawLine 580,411,775,411
247
248//      ShowTools/A
249        Button button0,pos={14,87},size={150,20},proc=LoadEventLog_Button,title="Load Event Log File"
250        Button button0,fSize=12
251        TitleBox tb1,pos={475,500},size={266,86},fSize=10
252        TitleBox tb1,variable= root:Packages:NIST:Event:gEventDisplayString
253
254        CheckBox chkbox2,pos={376,151},size={81,15},proc=LogIntEvent_Proc,title="Log Intensity"
255        CheckBox chkbox2,fSize=10,variable= root:Packages:NIST:Event:gEvent_logint
256        CheckBox chkbox3,pos={14,125},size={119,15},title="Remove Bad Events?",fSize=10
257        CheckBox chkbox3,variable= root:Packages:NIST:Event:gRemoveBadEvents
258       
259        Button doneButton,pos={738,36},size={50,20},proc=EventDone_Proc,title="Done"
260        Button doneButton,fSize=12
261        Button button2,pos={486,200},size={140,20},proc=ShowEventDataButtonProc,title="Show Event Data"
262        Button button3,pos={486,228},size={140,20},proc=ShowBinDetailsButtonProc,title="Show Bin Details"
263        Button button5,pos={633,228},size={140,20},proc=ExportSlicesButtonProc,title="Export Slices as VAX"
264        Button button6,pos={748,9},size={40,20},proc=EventModeHelpButtonProc,title="?"
265               
266        Button button7,pos={211,33},size={120,20},proc=AdjustEventDataButtonProc,title="Adjust Events"
267        Button button8,pos={653,201},size={120,20},proc=CustomBinButtonProc,title="Custom Bins"
268        Button button4,pos={211,63},size={120,20},proc=UndoTimeSortButtonProc,title="Undo Time Sort"
269        Button button18,pos={211,90},size={120,20},proc=EC_ImportWavesButtonProc,title="Import Edited"
270       
271        SetVariable setvar0,pos={208,149},size={160,16},proc=sliceSelectEvent_Proc,title="Display Time Slice"
272        SetVariable setvar0,fSize=10
273        SetVariable setvar0,limits={0,1000,1},value= root:Packages:NIST:Event:gEvent_tsdisp     
274        SetVariable setvar1,pos={389,29},size={160,16},title="Number of slices",fSize=10
275        SetVariable setvar1,limits={1,1000,1},value= root:Packages:NIST:Event:gEvent_nslices
276        SetVariable setvar2,pos={389,54},size={160,16},title="Max Time (s)",fSize=10
277        SetVariable setvar2,value= root:Packages:NIST:Event:gEvent_t_longest
278       
279        PopupMenu popup0,pos={389,77},size={119,20},proc=BinTypePopMenuProc,title="Bin Spacing"
280        PopupMenu popup0,fSize=10
281        PopupMenu popup0,mode=1,popvalue="Equal",value= #"\"Equal;Fibonacci;Custom;\""
282        Button button1,pos={389,103},size={120,20},fSize=12,proc=ProcessEventLog_Button,title="Bin Event Data"
283
284        Button button10,pos={488,305},size={100,20},proc=SplitFileButtonProc,title="Split Big File"
285        Button button14,pos={488,350},size={120,20},proc=Stream_LoadDecim,title="Load+Decimate"
286        Button button19,pos={639,350},size={130,20},proc=Stream_LoadAdjustedList,title="Load+Concatenate"
287        Button button20,pos={680,305},size={90,20},proc=ShowList_ToLoad,title="Show List"
288
289        Button button21,pos={649,378},size={120,20},proc=Stream_LoadAdjList_BinOnFly,title="Load+Accumulate"
290
291        SetVariable setvar3,pos={487,378},size={150,16},title="Decimation factor"
292        SetVariable setvar3,fSize=10
293        SetVariable setvar3,limits={1,inf,1},value= root:Packages:NIST:Event:gDecimation
294
295        Button button15_0,pos={488,425},size={110,20},proc=AccumulateSlicesButton,title="Add First Slice"
296        Button button16_1,pos={488,450},size={110,20},proc=AccumulateSlicesButton,title="Add Next Slice"
297        Button button17_2,pos={620,425},size={110,20},proc=AccumulateSlicesButton,title="Display Total"
298        Button button22,pos={620,450},size={120,20},proc=Osc_LoadAdjList_BinOnFly,title="Load+Accumulate"
299
300        CheckBox chkbox1_0,pos={25,34},size={69,14},title="Oscillatory",fSize=10
301        CheckBox chkbox1_0,mode=1,proc=EventModeRadioProc,value=1
302        CheckBox chkbox1_1,pos={25,59},size={53,14},title="Stream",fSize=10
303        CheckBox chkbox1_1,proc=EventModeRadioProc,value=0,mode=1
304        CheckBox chkbox1_2,pos={104,59},size={53,14},title="TISANE",fSize=10
305        CheckBox chkbox1_2,proc=EventModeRadioProc,value=0,mode=1
306        CheckBox chkbox1_3,pos={104,34},size={37,14},title="TOF",fSize=10
307        CheckBox chkbox1_3,proc=EventModeRadioProc,value=0,mode=1
308       
309        GroupBox group0_0,pos={5,5},size={174,112},title="(1) Loading Mode",fSize=12,fStyle=1
310        GroupBox group0_1,pos={372,5},size={192,127},title="(3) Bin Events",fSize=12,fStyle=1
311        GroupBox group0_2,pos={477,169},size={310,92},title="(4) View / Export",fSize=12,fStyle=1
312        GroupBox group0_3,pos={191,5},size={165,117},title="(2) Edit Events",fSize=12,fStyle=1
313        GroupBox group0_4,pos={474,278},size={312,200},title="Split / Accumulate Files",fSize=12
314        GroupBox group0_4,fStyle=1
315       
316        Display/W=(10,170,460,610)/HOST=#
317        AppendImage/T/G=1 :Packages:NIST:Event:dispsliceData            //  /G=1 flag prevents interpretation as RGB so 3, 4 slices display correctly
318        ModifyImage dispsliceData ctab= {*,*,ColdWarm,0}
319        ModifyImage dispsliceData ctabAutoscale=3
320        ModifyGraph margin(left)=14,margin(bottom)=14,margin(top)=14,margin(right)=14
321        ModifyGraph mirror=2
322        ModifyGraph nticks=4
323        ModifyGraph minor=1
324        ModifyGraph fSize=9
325        ModifyGraph standoff=0
326        ModifyGraph tkLblRot(left)=90
327        ModifyGraph btLen=3
328        ModifyGraph tlOffset=-2
329        RenameWindow #,Event_slicegraph
330        SetActiveSubwindow ##
331EndMacro
332
333
334
335
336// mode selector
337//Static Constant MODE_STREAM = 0
338//Static Constant MODE_OSCILL = 1
339//Static Constant MODE_TISANE = 2
340//Static Constant MODE_TOF = 3
341//
342Function EventModeRadioProc(name,value)
343        String name
344        Variable value
345       
346        NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
347       
348        strswitch (name)
349                case "chkbox1_0":
350                        gEventModeRadioVal= MODE_OSCILL
351                        break
352                case "chkbox1_1":
353                        gEventModeRadioVal= MODE_STREAM
354                        break
355                case "chkbox1_2":
356                        gEventModeRadioVal= MODE_TISANE
357                        break
358                case "chkbox1_3":
359                        gEventModeRadioVal= MODE_TOF
360                        break
361        endswitch
362        CheckBox chkbox1_0,value= gEventModeRadioVal==MODE_OSCILL
363        CheckBox chkbox1_1,value= gEventModeRadioVal==MODE_STREAM
364        CheckBox chkbox1_2,value= gEventModeRadioVal==MODE_TISANE
365        CheckBox chkbox1_3,value= gEventModeRadioVal==MODE_TOF
366
367        return(0)
368End
369
370Function AdjustEventDataButtonProc(ba) : ButtonControl
371        STRUCT WMButtonAction &ba
372
373        switch( ba.eventCode )
374                case 2: // mouse up
375                        // click code here
376                        Execute "ShowEventCorrectionPanel()"
377                        //
378                        break
379                case -1: // control being killed
380                        break
381        endswitch
382
383        return 0
384End
385
386Function CustomBinButtonProc(ba) : ButtonControl
387        STRUCT WMButtonAction &ba
388
389        switch( ba.eventCode )
390                case 2: // mouse up
391                        // click code here
392                        Execute "Show_CustomBinPanel()"
393                        //
394                        break
395                case -1: // control being killed
396                        break
397        endswitch
398
399        return 0
400End
401
402
403Function ShowEventDataButtonProc(ba) : ButtonControl
404        STRUCT WMButtonAction &ba
405
406        switch( ba.eventCode )
407                case 2: // mouse up
408                        // click code here
409                        Execute "ShowRescaledTimeGraph()"
410                        //
411                        DifferentiatedTime()
412                        //
413                        break
414                case -1: // control being killed
415                        break
416        endswitch
417
418        return 0
419End
420
421Function BinTypePopMenuProc(pa) : PopupMenuControl
422        STRUCT WMPopupAction &pa
423
424        switch( pa.eventCode )
425                case 2: // mouse up
426                        Variable popNum = pa.popNum
427                        String popStr = pa.popStr
428                        if(cmpstr(popStr,"Custom")==0)
429                                Execute "Show_CustomBinPanel()"
430                        endif
431                        break
432                case -1: // control being killed
433                        break
434        endswitch
435
436        return 0
437End
438
439Function ShowBinDetailsButtonProc(ba) : ButtonControl
440        STRUCT WMButtonAction &ba
441
442        switch( ba.eventCode )
443                case 2: // mouse up
444                        // click code here
445                        Execute "ShowBinTable()"
446                        Execute "BinEventBarGraph()"
447                        break
448                case -1: // control being killed
449                        break
450        endswitch
451
452        return 0
453End
454
455Function UndoTimeSortButtonProc(ba) : ButtonControl
456        STRUCT WMButtonAction &ba
457
458        switch( ba.eventCode )
459                case 2: // mouse up
460                        // click code here
461                        Execute "UndoTheSorting()"
462                        break
463                case -1: // control being killed
464                        break
465        endswitch
466
467        return 0
468End
469
470Function ExportSlicesButtonProc(ba) : ButtonControl
471        STRUCT WMButtonAction &ba
472
473        switch( ba.eventCode )
474                case 2: // mouse up
475                        // click code here
476                        Execute "ExportSlicesAsVAX()"           //will invoke the dialog
477                        break
478                case -1: // control being killed
479                        break
480        endswitch
481
482        return 0
483End
484
485Function EventModeHelpButtonProc(ba) : ButtonControl
486        STRUCT WMButtonAction &ba
487
488        switch( ba.eventCode )
489                case 2: // mouse up
490                        // click code here
491                        DisplayHelpTopic/Z "Event Mode Data"
492                        break
493                case -1: // control being killed
494                        break
495        endswitch
496
497        return 0
498End
499
500
501Function EventDone_Proc(ba) : ButtonControl
502        STRUCT WMButtonAction &ba
503       
504        String win = ba.win
505        switch (ba.eventCode)
506                case 2:
507                        DoWindow/K EventModePanel
508                        break
509        endswitch
510        return(0)
511End
512
513
514
515Function ProcessEventLog_Button(ctrlName) : ButtonControl
516        String ctrlName
517       
518        NVAR mode=root:Packages:NIST:Event:gEvent_Mode
519       
520        if(mode == MODE_STREAM)
521                Stream_ProcessEventLog("")
522        endif
523       
524        if(mode == MODE_OSCILL)
525                Osc_ProcessEventLog("")
526        endif
527       
528        // If TOF mode, process as Oscillatory -- that is, take the times as is
529        if(mode == MODE_TOF)
530                Osc_ProcessEventLog("")
531        endif
532       
533        // toggle the checkbox for log display to force the display to be correct
534        NVAR gLog = root:Packages:NIST:Event:gEvent_logint
535        LogIntEvent_Proc("",gLog)
536       
537        return(0)
538end
539
540// for oscillatory mode
541//
542Function Osc_ProcessEventLog(ctrlName)
543        String ctrlName
544
545        Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
546       
547        Wave binnedData = root:Packages:NIST:Event:binnedData
548        Wave xLoc = root:Packages:NIST:Event:xLoc
549        Wave yLoc = root:Packages:NIST:Event:yLoc
550
551// now with the number of slices and max time, process the events
552
553        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
554        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
555
556        SetDataFolder root:Packages:NIST:Event          //don't count on the folder remaining here
557       
558        Make/D/O/N=(XBINS,YBINS,nslices) slicedData
559               
560        Wave slicedData = slicedData
561        Wave rescaledTime = rescaledTime
562        Wave timePt = timePt
563        Make/O/D/N=(128,128) tmpData
564        Make/O/D/N=(nslices+1) binEndTime,binCount
565        Make/O/D/N=(nslices) timeWidth
566        Wave timeWidth = timeWidth
567        Wave binEndTime = binEndTime
568        Wave binCount = binCount
569
570        variable ii,del,p1,p2,t1,t2
571        del = t_longest/nslices
572
573        slicedData = 0
574        binEndTime[0]=0
575        BinCount[nslices]=0
576
577
578        String binTypeStr=""
579        ControlInfo /W=EventModePanel popup0
580        binTypeStr = S_value
581       
582        strswitch(binTypeStr)   // string switch
583                case "Equal":           // execute if case matches expression
584                        SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
585                        break                                           // exit from switch
586                case "Fibonacci":               // execute if case matches expression
587                        SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
588                        break
589                case "Log":             // execute if case matches expression
590                        SetLogBins(binEndTime,timeWidth,nslices,t_longest)
591                        break
592                case "Custom":          // execute if case matches expression
593                        //bins are set by the user on the panel - assume it's good to go
594                        break
595                default:                                                        // optional default expression executed
596                        DoAlert 0,"No match for bin type, Equal bins used"
597                        SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
598        endswitch
599
600
601// now before binning, sort the data
602
603        //MakeIndex is REALLY SLOW - this is slow - to allow undoing the sorting and starting over, but if you don't,
604        // you'll never be able to undo the sort
605        //
606        //
607        // for a 500 MB hst, MakeIndex = 83s, everything else in this function totalled < 5s
608        //
609        SetDataFolder root:Packages:NIST:Event:
610
611        if(WaveExists($"root:Packages:NIST:Event:OscSortIndex") == 0 )
612                Duplicate/O rescaledTime OscSortIndex
613//              tic()
614                MakeIndex rescaledTime OscSortIndex
615//              toc()
616//              tic()
617                IndexSort OscSortIndex, yLoc,xLoc,timePt,rescaledTime   
618//              toc()
619                //SetDataFolder root:Packages:NIST:Event
620//              tic()
621                IndexForHistogram(xLoc,yLoc,binnedData)                 // index the events AFTER sorting
622//              toc()
623                //SetDataFolder root:
624        Endif
625       
626        Wave index = root:Packages:NIST:Event:SavedIndex                //this is the histogram index
627
628//      tic()
629        for(ii=0;ii<nslices;ii+=1)
630                if(ii==0)
631//                      t1 = ii*del
632//                      t2 = (ii+1)*del
633                        p1 = BinarySearch(rescaledTime,0)
634                        p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
635                else
636//                      t2 = (ii+1)*del
637                        p1 = p2+1               //one more than the old one
638                        p2 = BinarySearch(rescaledTime,binEndTime[ii+1])               
639                endif
640
641        // typically zero will never be a valid time value in oscillatory mode. in "stream" mode, the first is normalized to == 0
642        // but not here - times are what they are.
643                if(p1 == -1)
644                        Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
645                        p1 = 0          //set to the first point if it's off the end
646                Endif
647               
648                if(p2 == -2)
649                        Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
650                        p2 = numpnts(rescaledTime)-1            //set to the last point if it's off the end
651                Endif
652//              Print p1,p2
653
654
655                tmpData=0
656                JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
657                slicedData[][][ii] = tmpData[p][q]
658               
659//              binEndTime[ii+1] = t2
660                binCount[ii] = sum(tmpData,-inf,inf)
661        endfor
662
663//      toc()
664        Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
665        Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
666        logslicedData = log(slicedData)
667
668        SetDataFolder root:
669        return(0)
670End
671
672// for a "continuous exposure"
673//
674// if there is a sort of these events, I need to re-index the events for the histogram
675// - see the oscillatory mode  - and sort the events here, then immediately re-index for the histogram
676// - but with the added complication that I need to always remember to index for the histogram, every time
677// - since I don't know if I've sorted or un-sorted. Osc mode always forces a re-sort and a re-index
678//
679Function Stream_ProcessEventLog(ctrlName)
680        String ctrlName
681
682//      NVAR slicewidth = root:Packages:NIST:gTISANE_slicewidth
683
684       
685        Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
686       
687        Wave binnedData = root:Packages:NIST:Event:binnedData
688        Wave xLoc = root:Packages:NIST:Event:xLoc
689        Wave yLoc = root:Packages:NIST:Event:yLoc
690
691// now with the number of slices and max time, process the events
692
693        NVAR yesSortStream = root:Packages:NIST:Event:gSortStreamEvents         //do I sort the events?
694        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
695        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
696
697        SetDataFolder root:Packages:NIST:Event          //don't count on the folder remaining here
698       
699        Make/D/O/N=(XBINS,YBINS,nslices) slicedData
700               
701        Wave slicedData = slicedData
702        Wave rescaledTime = rescaledTime
703        Make/O/D/N=(128,128) tmpData
704        Make/O/D/N=(nslices+1) binEndTime,binCount//,binStartTime
705        Make/O/D/N=(nslices) timeWidth
706        Wave binEndTime = binEndTime
707        Wave timeWidth = timeWidth
708        Wave binCount = binCount
709
710        variable ii,del,p1,p2,t1,t2
711        del = t_longest/nslices
712
713        slicedData = 0
714        binEndTime[0]=0
715        BinCount[nslices]=0
716       
717        String binTypeStr=""
718        ControlInfo /W=EventModePanel popup0
719        binTypeStr = S_value
720       
721        strswitch(binTypeStr)   // string switch
722                case "Equal":           // execute if case matches expression
723                        SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
724                        break                                           // exit from switch
725                case "Fibonacci":               // execute if case matches expression
726                        SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
727                        break
728                case "Log":             // execute if case matches expression
729                        SetLogBins(binEndTime,timeWidth,nslices,t_longest)
730                        break
731                case "Custom":          // execute if case matches expression
732                        //bins are set by the user on the panel - assume it's good to go
733                        break
734                default:                                                        // optional default expression executed
735                        DoAlert 0,"No match for bin type, Equal bins used"
736                        SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
737        endswitch
738
739// TODO
740// the global exists for this switch, but it is not implemented - not sure whether
741// it's correct to implement this at all --
742//
743        if(yesSortStream == 1)
744                SortTimeData()
745        endif
746       
747// index the events before binning
748// if there is a sort of these events, I need to re-index the events for the histogram
749//      SetDataFolder root:Packages:NIST:Event
750        IndexForHistogram(xLoc,yLoc,binnedData)
751//      SetDataFolder root:
752        Wave index = root:Packages:NIST:Event:SavedIndex                //the index for the histogram
753       
754       
755        for(ii=0;ii<nslices;ii+=1)
756                if(ii==0)
757//                      t1 = ii*del
758//                      t2 = (ii+1)*del
759                        p1 = BinarySearch(rescaledTime,0)
760                        p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
761                else
762//                      t2 = (ii+1)*del
763                        p1 = p2+1               //one more than the old one
764                        p2 = BinarySearch(rescaledTime,binEndTime[ii+1])               
765                endif
766
767                if(p1 == -1)
768                        Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
769                        p1 = 0          //set to the first point if it's off the end
770                Endif
771                if(p2 == -2)
772                        Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
773                        p2 = numpnts(rescaledTime)-1            //set to the last point if it's off the end
774                Endif
775//              Print p1,p2
776
777
778                tmpData=0
779                JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
780                slicedData[][][ii] = tmpData[p][q]
781               
782//              binEndTime[ii+1] = t2
783                binCount[ii] = sum(tmpData,-inf,inf)
784        endfor
785
786        Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
787        Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
788        logslicedData = log(slicedData)
789
790        SetDataFolder root:
791        return(0)
792End
793
794
795Proc    UndoTheSorting()
796        Osc_UndoSort()
797End
798
799// for oscillatory mode
800//
801// -- this takes the previously generated index, and un-sorts the data to restore to the
802// "as-collected" state
803//
804Function Osc_UndoSort()
805
806        SetDataFolder root:Packages:NIST:Event          //don't count on the folder remaining here
807        Wave rescaledTime = rescaledTime
808        Wave OscSortIndex = OscSortIndex
809        Wave yLoc = yLoc
810        Wave xLoc = xLoc
811        Wave timePt = timePt
812
813        Sort OscSortIndex OscSortIndex,yLoc,xLoc,timePt,rescaledTime
814
815        KillWaves/Z OscSortIndex
816       
817        SetDataFolder root:
818        return(0)
819End
820
821
822// now before binning, sort the data
823//
824//this is slow - undoing the sorting and starting over, but if you don't,
825// you'll never be able to undo the sort
826//
827Function SortTimeData()
828
829
830        SetDataFolder root:Packages:NIST:Event:
831
832        KillWaves/Z OscSortIndex
833       
834        if(WaveExists($"root:Packages:NIST:Event:OscSortIndex") == 0 )
835                Duplicate/O rescaledTime OscSortIndex
836                MakeIndex rescaledTime OscSortIndex
837                IndexSort OscSortIndex, yLoc,xLoc,timePt,rescaledTime   
838        Endif
839       
840        SetDataFolder root:
841        return(0)
842End
843
844
845
846Function SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
847        Wave binEndTime,timeWidth
848        Variable nslices,t_longest
849
850        Variable del,ii,t2
851        binEndTime[0]=0         //so the bar graph plots right...
852        del = t_longest/nslices
853       
854        for(ii=0;ii<nslices;ii+=1)
855                t2 = (ii+1)*del
856                binEndTime[ii+1] = t2
857        endfor
858        binEndTime[ii+1] = t_longest*(1-1e-6)           //otherwise floating point errors such that the last time point is off the end of the Binary search
859
860        timeWidth = binEndTime[p+1]-binEndTime[p]
861
862        return(0)       
863End
864
865// TODO
866// either get this to work, or scrap it entirely. it currently isn't on the popup
867// so it can't be accessed
868Function SetLogBins(binEndTime,timeWidth,nslices,t_longest)
869        Wave binEndTime,timeWidth
870        Variable nslices,t_longest
871
872        Variable tMin,ii
873
874        Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
875       
876        binEndTime[0]=0         //so the bar graph plots right...
877
878        // just like the log-scaled q-points
879        tMin = rescaledTime[1]/1                        //just a guess... can't use tMin=0, and rescaledTime[0] == 0 by definition
880        Print rescaledTime[1], tMin
881        for(ii=0;ii<nslices;ii+=1)
882                binEndTime[ii+1] =alog(log(tMin) + (ii+1)*((log(t_longest)-log(tMin))/nslices))
883        endfor
884        binEndTime[ii+1] = t_longest            //otherwise floating point errors such that the last time point is off the end of the Binary search
885       
886        timeWidth = binEndTime[p+1]-binEndTime[p]
887
888        return(0)
889End
890
891Function MakeFibonacciWave(w,num)
892        Wave w
893        Variable num
894
895        //skip the initial zero
896        Variable f1,f2,ii
897        f1=1
898        f2=1
899        w[0] = f1
900        w[1] = f2
901        for(ii=2;ii<num;ii+=1)
902                w[ii] = f1+f2
903                f1=f2
904                f2=w[ii]
905        endfor
906               
907        return(0)
908end
909
910Function SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
911        Wave binEndTime,timeWidth
912        Variable nslices,t_longest
913
914        Variable tMin,ii,total,t2,tmp
915        Make/O/D/N=(nslices) fibo
916        fibo=0
917        MakeFibonacciWave(fibo,nslices)
918       
919//      Make/O/D tmpFib={1,1,2,3,5,8,13,21,34,55,89,144,233,377,610,987,1597,2584,4181,6765,10946}
920
921        binEndTime[0]=0         //so the bar graph plots right...
922        total = sum(fibo,0,nslices-1)           //total number of "pieces"
923       
924        tmp=0
925        for(ii=0;ii<nslices;ii+=1)
926                t2 = sum(fibo,0,ii)/total*t_longest
927                binEndTime[ii+1] = t2
928        endfor
929        binEndTime[ii+1] = t_longest            //otherwise floating point errors such that the last time point is off the end of the Binary search
930       
931        timeWidth = binEndTime[p+1]-binEndTime[p]
932       
933        return(0)
934End
935
936
937
938// TODO:
939//
940// ** currently, the "stream" loader uses the first data point as time=0
941//    and rescales everything to that time. "Osc" loading uses the times "as-is"
942//    from the file, trusting the times to be correct.
943//
944// Would TISANE or TOF need a different loader?
945//     
946Function LoadEventLog_Button(ctrlName) : ButtonControl
947        String ctrlName
948
949        NVAR mode=root:Packages:NIST:Event:gEvent_mode
950        Variable err=0
951        Variable fileref,totBytes
952        NVAR fileTooLarge = root:Packages:NIST:Event:gEventFileTooLarge         //limit load to 150MB
953
954        SVAR filename = root:Packages:NIST:Event:gEvent_logfile
955        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
956        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
957       
958        String fileFilters = "All Files:.*;Data Files (*.txt):.txt;"
959        String abortStr
960       
961        PathInfo/S catPathName
962        if(V_flag==0)
963                DoAlert 0,"Please 'Pick Path' to the data from the Main (yellow) Panel."
964                return(0)
965        endif
966       
967       
968        Open/R/D/P=catPathName/F=fileFilters fileref
969        filename = S_filename
970        if(strlen(S_filename) == 0)
971                // user cancelled
972                DoAlert 0,"No file selected, no file loaded."
973                return(1)
974        endif
975       
976/// Abort if the files are too large
977        Open/R fileref as fileName
978                FStatus fileref
979        Close fileref
980
981        totBytes = V_logEOF/1e6         //in MB
982        if(totBytes > fileTooLarge)
983                sprintf abortStr,"File is %g MB, larger than the limit of %g MB. Split and Decimate.",totBytes,fileTooLarge
984                Abort abortStr
985        endif
986       
987
988#if (exists("EventLoadWave")==4)
989        LoadEvents_XOP()
990#else
991        // XOP is not present, warn the user to re-run the installer
992        //check the 32-bit or 64-bit
993        String igorKindStr = StringByKey("IGORKIND", IgorInfo(0) )
994        String alertStr
995        if(strsearch(igorKindStr, "64", 0 ) != -1)
996                alertStr = "The Event Loader XOP is not installed for the 64-bit version of Igor. Without it, event loading will "
997                alertStr += "be slow. It is recommended that you re-run the NCNR Installer. Click NO to stop and "
998                alertStr += "do the installation, or YES to continue with the file loading."
999        else
1000                alertStr = "The Event Loader XOP is not installed for the 32-bit version of Igor. Without it, event loading will "
1001                alertStr += "be slow. It is recommended that you re-run the NCNR Installer. Click NO to stop and "
1002                alertStr += "do the installation, or YES to continue with the file loading."
1003        endif
1004        DoAlert 1,alertStr
1005       
1006        if(V_flag == 0)
1007                // get out gracefully
1008                SetDataFolder root:
1009                return(0)
1010        endif
1011
1012        LoadEvents_New_noXOP()
1013//      LoadEvents()
1014
1015       
1016#endif 
1017
1018        SetDataFolder root:Packages:NIST:Event:
1019
1020//tic()
1021        Wave timePt=timePt
1022        Wave xLoc=xLoc
1023        Wave yLoc=yLoc
1024        CleanupTimes(xLoc,yLoc,timePt)          //remove zeroes
1025//toc()
1026       
1027       
1028/////
1029// now do a little processing of the times based on the type of data
1030//     
1031        if(mode == MODE_STREAM)         // continuous "Stream" mode - start from zero
1032                Duplicate/O timePt rescaledTime
1033                MultiThread rescaledTime = 1e-7*(timePt-timePt[0])              //convert to seconds and start from zero
1034                t_longest = waveMax(rescaledTime)               //should be the last point     
1035        endif
1036       
1037        if(mode == MODE_OSCILL)         // oscillatory mode - don't adjust the times, we get periodic t0 to reset t=0
1038                Duplicate/O timePt rescaledTime
1039                MultiThread rescaledTime *= 1e-7                        //convert to seconds and that's all
1040                t_longest = waveMax(rescaledTime)               //if oscillatory, won't be the last point, so get it this way
1041       
1042                KillWaves/Z OscSortIndex                        //to make sure that there is no old index hanging around
1043        endif
1044
1045// MODE_TISANE
1046//TODO -- tisane doesn't do anything different than oscillatory. Is this really correct??
1047        if(mode == MODE_TISANE)         // TISANE mode - don't adjust the times, we get periodic t0 to reset t=0
1048                Duplicate/O timePt rescaledTime
1049                MultiThread rescaledTime *= 1e-7                        //convert to seconds and that's all
1050                t_longest = waveMax(rescaledTime)               //if oscillatory, won't be the last point, so get it this way
1051       
1052                KillWaves/Z OscSortIndex                        //to make sure that there is no old index hanging around
1053        endif
1054       
1055// MODE_TOF
1056        if(mode == MODE_TOF)            // TOF mode - don't adjust the times, we get periodic t0 to reset t=0
1057                Duplicate/O timePt rescaledTime
1058                MultiThread rescaledTime *= 1e-7                        //convert to seconds and that's all
1059                t_longest = waveMax(rescaledTime)               //if oscillatory, won't be the last point, so get it this way
1060       
1061                KillWaves/Z OscSortIndex                        //to make sure that there is no old index hanging around
1062        endif
1063
1064        SetDataFolder root:
1065
1066        STRUCT WMButtonAction ba
1067        ba.eventCode = 2
1068        ShowEventDataButtonProc(ba)
1069
1070        SetDataFolder root:
1071
1072        return(0)
1073End
1074
1075//// for the mode of "one continuous exposure"
1076////
1077//Function Stream_LoadEventLog(ctrlName)
1078//      String ctrlName
1079//     
1080//      Variable fileref
1081//
1082//      SVAR filename = root:Packages:NIST:Event:gEvent_logfile
1083//      NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
1084//      NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1085//     
1086//      String fileFilters = "All Files:.*;Data Files (*.txt):.txt;"
1087//     
1088//      Open/R/D/F=fileFilters fileref
1089//      filename = S_filename
1090//      if(strlen(S_filename) == 0)
1091//              // user cancelled
1092//              DoAlert 0,"No file selected, no file loaded."
1093//              return(1)
1094//      endif
1095//
1096//#if (exists("EventLoadWave")==4)
1097//      LoadEvents_XOP()
1098//#else
1099//      LoadEvents()
1100//#endif       
1101//
1102//      SetDataFolder root:Packages:NIST:Event:
1103//
1104////tic()
1105//      Wave timePt=timePt
1106//      Wave xLoc=xLoc
1107//      Wave yLoc=yLoc
1108//      CleanupTimes(xLoc,yLoc,timePt)          //remove zeroes
1109//     
1110////toc()
1111//
1112//      Duplicate/O timePt rescaledTime
1113//      rescaledTime = 1e-7*(timePt-timePt[0])          //convert to seconds and start from zero
1114//      t_longest = waveMax(rescaledTime)               //should be the last point
1115//
1116//      SetDataFolder root:
1117//
1118//      return(0)
1119//End
1120//
1121//// for the mode "oscillatory"
1122////
1123//Function Osc_LoadEventLog(ctrlName)
1124//      String ctrlName
1125//     
1126//      Variable fileref
1127//
1128//      SVAR filename = root:Packages:NIST:Event:gEvent_logfile
1129//      NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
1130//      NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1131//     
1132//      String fileFilters = "All Files:.*;Data Files (*.txt):.txt;"
1133//     
1134//      Open/R/D/F=fileFilters fileref
1135//      filename = S_filename
1136//              if(strlen(S_filename) == 0)
1137//              // user cancelled
1138//              DoAlert 0,"No file selected, no file loaded."
1139//              return(1)
1140//      endif
1141//     
1142//#if (exists("EventLoadWave")==4)
1143//      LoadEvents_XOP()
1144//#else
1145//      LoadEvents()
1146//#endif       
1147//     
1148//      SetDataFolder root:Packages:NIST:Event:
1149//
1150//      Wave timePt=timePt
1151//      Wave xLoc=xLoc
1152//      Wave yLoc=yLoc
1153//      CleanupTimes(xLoc,yLoc,timePt)          //remove zeroes
1154//     
1155//      Duplicate/O timePt rescaledTime
1156//      rescaledTime *= 1e-7                    //convert to seconds and that's all
1157//      t_longest = waveMax(rescaledTime)               //if oscillatory, won't be the last point, so get it this way
1158//
1159//      KillWaves/Z OscSortIndex                        //to make sure that there is no old index hanging around
1160//
1161//      SetDataFolder root:
1162//
1163//      return(0)
1164//End
1165
1166
1167//
1168// -- MUCH faster to count the number of lines to remove, then delete (N)
1169// rather then delete them one-by-one in the do-loop
1170Function CleanupTimes(xLoc,yLoc,timePt)
1171        Wave xLoc,yLoc,timePt
1172
1173        // start at the back and remove zeros
1174        Variable num=numpnts(xLoc),ii,numToRemove
1175
1176        numToRemove = 0
1177        ii=num
1178        do
1179                ii -= 1
1180                if(timePt[ii] == 0 && xLoc[ii] == 0 && yLoc[ii] == 0)
1181                        numToRemove += 1
1182                endif
1183        while(timePt[ii-1] == 0 && xLoc[ii-1] == 0 && yLoc[ii-1] == 0)
1184       
1185        if(numToRemove != 0)
1186                DeletePoints ii, numToRemove, xLoc,yLoc,timePt
1187        endif
1188       
1189        return(0)
1190End
1191
1192Function LogIntEvent_Proc(ctrlName,checked) : CheckBoxControl
1193        String ctrlName
1194        Variable checked
1195               
1196        SetDataFolder root:Packages:NIST:Event
1197       
1198        Wave slicedData = slicedData
1199        Wave logSlicedData = logSlicedData
1200        Wave dispSliceData = dispSliceData
1201       
1202        if(checked)
1203                logslicedData = log(slicedData)
1204                Duplicate/O logslicedData dispsliceData
1205        else
1206                Duplicate/O slicedData dispsliceData
1207        endif
1208
1209        NVAR selectedslice = root:Packages:NIST:Event:gEvent_tsdisp
1210
1211        sliceSelectEvent_Proc("", selectedslice, "", "")
1212
1213        SetDataFolder root:
1214
1215End
1216
1217
1218// TODO (DONE)
1219// this "fails" for data sets that have 3 or 4 slices, as the ModifyImage command
1220// interprets the data as being RGB - and so does nothing.
1221// need to find a way around this
1222//
1223////  When first plotted, AppendImage/G=1 flag prevents interpretation as RGB so 3, 4 slices display correctly
1224///
1225// I could modify this procedure to use the log = 0|1 keyword for the log Z display
1226// rather than creating a duplicate wave of log(data)
1227//
1228Function sliceSelectEvent_Proc(ctrlName, varNum, varStr, varName) : SetVariableControl
1229        String ctrlName
1230        Variable varNum
1231        String varStr
1232        String varName
1233       
1234        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
1235        NVAR selectedslice = root:Packages:NIST:Event:gEvent_tsdisp
1236       
1237        if(varNum < 0)
1238                selectedslice = 0
1239                DoUpdate
1240        elseif (varNum > nslices-1)
1241                selectedslice = nslices-1
1242                DoUpdate
1243        else
1244                ModifyImage/W=EventModePanel#Event_slicegraph ''#0 plane = varNum
1245        endif
1246
1247End
1248
1249Function DifferentiatedTime()
1250
1251        Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
1252
1253        SetDataFolder root:Packages:NIST:Event:
1254               
1255        Differentiate rescaledTime/D=rescaledTime_DIF
1256//      Display rescaledTime,rescaledTime_DIF
1257        DoWindow/F Differentiated_Time
1258        if(V_flag == 0)
1259                Display/N=Differentiated_Time/K=1 rescaledTime_DIF
1260                Legend
1261                Modifygraph gaps=0
1262                ModifyGraph zero(left)=1
1263                Label left "\\Z14Delta (dt/event)"
1264                Label bottom "\\Z14Event number"
1265        endif
1266       
1267        SetDataFolder root:
1268       
1269        return(0)
1270End
1271
1272
1273//
1274// for the bit shifts, see the decimal-binary conversion
1275// http://www.binaryconvert.com/convert_unsigned_int.html
1276//
1277//              K0 = 536870912
1278//              Print (K0 & 0x08000000)/134217728       //bit 27 only, shift by 2^27
1279//              Print (K0 & 0x10000000)/268435456               //bit 28 only, shift by 2^28
1280//              Print (K0 & 0x20000000)/536870912               //bit 29 only, shift by 2^29
1281//
1282// This is duplicated by the XOP, but the Igor code allows quick access to print out
1283// all of the gory details of the events and every little bit of them. the print
1284// statements and flags are kept for this reason, so the code is a bit messy.
1285//
1286Function LoadEvents_OLD()
1287       
1288        NVAR time_msw = root:Packages:NIST:Event:gEvent_time_msw
1289        NVAR time_lsw = root:Packages:NIST:Event:gEvent_time_lsw
1290        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1291       
1292        SVAR filepathstr = root:Packages:NIST:Event:gEvent_logfile
1293        SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
1294       
1295       
1296////    Variable decFac = 10                    //decimation factor
1297////    Variable jj,keep
1298       
1299        SetDataFolder root:Packages:NIST:Event
1300
1301        Variable fileref
1302        String buffer
1303        String fileStr,tmpStr
1304        Variable dataval,timeval,type,numLines,verbose,verbose3
1305        Variable xval,yval,rollBit,nRoll,roll_time,bit29,bit28,bit27
1306        Variable ii,flaggedEvent,rolloverHappened,numBad=0,tmpPP=0,tmpT0=0
1307        Variable Xmax, yMax
1308       
1309        xMax = 127              // number the detector from 0->127
1310        yMax = 127
1311       
1312        verbose3 = 0                    //prints out the rollover events (type==3)
1313        verbose = 0
1314        numLines = 0
1315
1316       
1317        // what I really need is the number of XY events
1318        Variable numXYevents,num1,num2,num3,num0,totBytes,numPP,numT0,numDL,numFF,numZero
1319        Variable numRemoved
1320        numXYevents = 0
1321        num0 = 0
1322        num1 = 0
1323        num2 = 0
1324        num3 = 0
1325        numPP = 0
1326        numT0 = 0
1327        numDL = 0
1328        numFF = 0
1329        numZero = 0
1330        numRemoved = 0
1331
1332//tic()
1333        Open/R fileref as filepathstr
1334                FStatus fileref
1335        Close fileref
1336
1337        totBytes = V_logEOF
1338        Print "total bytes = ", totBytes
1339        if(EventDataType(filePathStr) != 1)             //if not hst extension of raw data
1340                Abort "Data must be raw event data with file extension .hst for this operation"
1341                return(0)
1342        endif   
1343//toc()
1344//
1345
1346
1347// do a "pre-scan to get some of the counts, so that I can allocate space. This does
1348// double the read time, but is still faster than adding points to waves as the file is read
1349//     
1350
1351        tic()
1352
1353        Open/R fileref as filepathstr
1354        do
1355                do
1356                        FReadLine fileref, buffer                       //skip the "blank" lines that have one character
1357                while(strlen(buffer) == 1)             
1358
1359                if (strlen(buffer) == 0)
1360                        break
1361                endif
1362               
1363                sscanf buffer,"%x",dataval
1364               
1365                // two most sig bits (31-30)
1366                type = (dataval & 0xC0000000)/1073741824                //right shift by 2^30
1367                               
1368                if(type == 0)
1369                        num0 += 1
1370                        numXYevents += 1
1371                endif
1372                if(type == 2)
1373                        num2 += 1
1374                        numXYevents += 1
1375                endif
1376                if(type == 1)
1377                        num1 += 1
1378                endif
1379                if(type == 3)
1380                        num3 += 1
1381                endif   
1382               
1383                bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1384               
1385                if(type==0 || type==2)
1386                        numPP += round(bit29)
1387                endif
1388               
1389                if(type==1 || type==3)
1390                        numT0 += round(bit29)
1391                endif
1392               
1393                if(dataval == 0)
1394                        numZero += 1
1395                endif
1396               
1397        while(1)
1398        Close fileref
1399//              done counting the number of XY events
1400        printf("Igor pre-scan done in  ")
1401        toc()
1402       
1403
1404        Print "(Igor) numT0 = ",numT0   
1405        Print "num0 = ",num0   
1406        Print "num1 = ",num1   
1407        Print "num2 = ",num2   
1408        Print "num3 = ",num3   
1409
1410//
1411//     
1412//      Printf "numXYevents = %d\r",numXYevents
1413//      Printf "XY = num0 = %d\r",num0
1414//      Printf "XY time = num2 = %d\r",num2
1415//      Printf "time MSW = num1 = %d\r",num1
1416//      Printf "Rollover = num3 = %d\r",num3
1417//      Printf "num0 + num2 = %d\r",num0+num2
1418
1419// dispStr will be displayed on the panel
1420        fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
1421       
1422        sprintf tmpStr, "%s: %d total bytes\r",fileStr,totBytes
1423        dispStr = tmpStr
1424        sprintf tmpStr,"numXYevents = %d\r",numXYevents
1425        dispStr += tmpStr
1426        sprintf tmpStr,"PP = %d  :  ",numPP
1427        dispStr += tmpStr
1428        sprintf tmpStr,"ZeroData = %d\r",numZero
1429        dispStr += tmpStr
1430        sprintf tmpStr,"Rollover = %d",num3
1431        dispStr += tmpStr
1432
1433        // /I/U is unsigned 32-bit integer (for the time)
1434        // /B/U is unsigned 8-bit integer (max val=255) for the x and y values
1435       
1436        Make/O/U/N=(numXYevents) xLoc,yLoc
1437        Make/O/D/N=(numXYevents) timePt
1438////    Make/O/U/N=(numXYevents/decFac) xLoc,yLoc
1439////    Make/O/D/N=(numXYevents/decFac) timePt
1440//      Make/O/U/N=(totBytes/4) xLoc,yLoc               //too large, trim when done (bad idea)
1441//      Make/O/D/N=(totBytes/4) timePt
1442        Make/O/D/N=1000 badTimePt,badEventNum,PPTime,PPEventNum,T0Time,T0EventNum
1443        badTimePt=0
1444        badEventNum=0
1445        PPTime=0
1446        PPEventNum=0
1447        T0Time=0
1448        T0EventNum=0
1449        xLoc=0
1450        yLoc=0
1451        timePt=0
1452       
1453        nRoll = 0               //number of rollover events
1454        roll_time = 2^26                //units of 10-7 sec
1455       
1456        NVAR removeBadEvents = root:Packages:NIST:Event:gRemoveBadEvents
1457       
1458        time_msw=0
1459       
1460        tic()
1461       
1462        ii = 0          //indexes the points in xLoc,yLoc,timePt
1463////    keep = decFac           //keep the first point
1464       
1465       
1466        Open/R fileref as filepathstr
1467       
1468        // remove events at the beginning up to a type==2 so that the msw and lsw times are reset properly
1469        if(RemoveBadEvents == 1)
1470                do
1471                        do
1472                                FReadLine fileref, buffer                       //skip the "blank" lines that have one character
1473                        while(strlen(buffer) == 1)             
1474       
1475                        if (strlen(buffer) == 0)
1476                                break
1477                        endif
1478                       
1479                        sscanf buffer,"%x",dataval
1480                // two most sig bits (31-30)
1481                        type = (dataval & 0xC0000000)/1073741824                //right shift by 2^30
1482                       
1483                        if(type == 2)
1484                                // this is the first event with a proper time value, so process the XY-time event as ususal
1485                                // and then break to drop to the main loop, where the next event == type 1
1486                               
1487                                xval = xMax - (dataval & 255)                                           //last 8 bits (7-0)
1488                                yval = (dataval & 65280)/256                                            //bits 15-8, right shift by 2^8
1489               
1490                                time_lsw = (dataval & 536805376)/65536                  //13 bits, 28-16, right shift by 2^16
1491               
1492                                if(verbose)
1493                //                                      printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1494                                        printf "%u : %u : %u : %u\r",dataval,timeval,xval,yval
1495                                endif
1496                               
1497                                // this is the first point, be sure that ii = 0, and always keep this point
1498////                            if(keep==decFac)
1499                                        ii = 0
1500                                        xLoc[ii] = xval
1501                                        yLoc[ii] = yval
1502////                                    keep = 0
1503////                            endif
1504                                Print "At beginning of file, numBad = ",numBad
1505                                break   // the next do loop processes the bulk of the file (** the next event == type 1 = MIR)
1506                        else
1507                                numBad += 1
1508                                numRemoved += 1
1509                        endif
1510                       
1511                        //ii+=1         don't increment the counter
1512                while(1)
1513        endif
1514       
1515        // now read the main portion of the file.
1516////    // keep is = 0 if bad points were removed, or is decFac is I need to keep the first point
1517        do
1518                do
1519                        FReadLine fileref, buffer                       //skip the "blank" lines that have one character
1520                while(strlen(buffer) == 1)             
1521
1522                if (strlen(buffer) == 0)                                // this marks the end of the file and is our only way out
1523                        break
1524                endif
1525               
1526                sscanf buffer,"%x",dataval
1527               
1528
1529//              type = (dataval & ~(2^32 - 2^30 -1))/2^30
1530
1531                // two most sig bits (31-30)
1532                type = (dataval & 0xC0000000)/1073741824                //right shift by 2^30
1533               
1534                //
1535                // The defintions of the event types
1536                //
1537                //Constant ATXY = 0
1538                //Constant ATXYM = 2
1539                //Constant ATMIR = 1
1540                //Constant ATMAR = 3
1541                //
1542                                               
1543                if(verbose > 0)
1544                        verbose -= 1
1545                endif
1546//             
1547                switch(type)
1548                        case ATXY:              // 0
1549                                if(verbose)             
1550                                        printf "XY : "         
1551                                endif
1552                               
1553                                // if the datavalue is == 0, just skip it now (it can only be interpreted as type 0, obviously)
1554                                if(dataval == 0 && RemoveBadEvents == 1)
1555                                        numRemoved += 1
1556//                                      Print "zero at ii= ",ii
1557                                        break           //don't increment ii
1558                                endif
1559                               
1560                                // if it's a pileup event, skip it now (this can be either type 0 or 2)
1561                                bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1562                                if(bit29 == 1 && RemoveBadEvents == 1)
1563                                        PPTime[tmpPP] = timeval
1564                                        PPEventNum[tmpPP] = ii
1565                                        tmpPP += 1
1566                                        numRemoved += 1
1567                                        break           //don't increment ii
1568                                endif
1569                               
1570//                              xval = ~(dataval & ~(2^32 - 2^8)) & 127
1571//                              yval = ((dataval & ~(2^32 - 2^16 ))/2^8) & 127
1572//                              time_lsw = (dataval & ~(2^32 - 2^29))/2^16
1573
1574                                xval = xMax - (dataval & 255)                                           //last 8 bits (7-0)
1575                                yval = (dataval & 65280)/256                                            //bits 15-8, right shift by 2^8
1576                                time_lsw = (dataval & 536805376)/65536                  //13 bits, 28-16, right shift by 2^16
1577
1578                                timeval = trunc( nRoll*roll_time + (time_msw * (8192)) + time_lsw )             //left shift msw by 2^13, then add in lsw, as an integer
1579                                if (timeval > t_longest)
1580                                        t_longest = timeval
1581                                endif
1582                               
1583                               
1584                                // catch the "bad" events:
1585                                // if an XY event follows a rollover, time_msw is 0 by definition, but does not immediately get
1586                                // re-evalulated here. Throw out only the immediately following points where msw is still 8191
1587                                if(rolloverHappened && RemoveBadEvents == 1)
1588                                        // maybe a bad event
1589                                        if(time_msw == 8191)
1590                                                badTimePt[numBad] = timeVal
1591                                                badEventNum[numBad] = ii
1592                                                numBad +=1
1593                                                numRemoved += 1
1594                                        else
1595                                                // time_msw has been reset, points are good now, so keep this one
1596////                                            if(keep==decFac)
1597                                                        xLoc[ii] = xval
1598                                                        yLoc[ii] = yval
1599                                                        timePt[ii] = timeval
1600                                                       
1601        //                                              if(xval == 127 && yval == 0)
1602        //                                                      // check bit 29
1603        //                                                      bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1604        //                                                      Print "XY=127,0 : bit29 = ",bit29
1605        //                                              endif
1606                                                       
1607                                                        ii+=1
1608                                                        rolloverHappened = 0
1609////                                                    keep = 0
1610////                                            else
1611////                                                    keep += 1
1612////                                            endif
1613                                        endif
1614                                else
1615                                        // normal processing of good point, keep it
1616////                                    if(keep==decFac)
1617                                                xLoc[ii] = xval
1618                                                yLoc[ii] = yval
1619                                                timePt[ii] = timeval
1620                                       
1621        //                                      if(xval == 127 && yval == 0)
1622        //                                              // check bit 29
1623        //                                              bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1624        //                                              Printf "XY=127,0 : bit29 = %u : d=%u\r",bit29,dataval
1625        //                                      endif
1626                                                ii+=1
1627////                                            keep = 0
1628////                                    else
1629////                                            keep += 1
1630////                                    endif
1631                                endif
1632
1633
1634                                if(verbose)             
1635//                                      printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1636                                        printf "d=%u : t=%u : msw=%u : lsw=%u : %u : %u \r",dataval,timeval,time_msw,time_lsw,xval,yval
1637                                endif                           
1638       
1639//                              verbose = 0
1640                                break
1641                        case ATXYM: // 2
1642                                if(verbose)
1643                                        printf "XYM : "
1644                                endif
1645                               
1646                                // if it's a pileup event, skip it now (this can be either type 0 or 2)
1647                                // - but can I do this if this is an XY-time event? This will lead to a wrong time, and a time
1648                                // assigned to an XY (0,0)...
1649//                              bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1650//                              if(bit29 == 1 && RemoveBadEvents == 1)
1651//                                      Print "*****Bit 29 (PP) event set for Type==2, but not handled, ii = ",ii
1652////                                    break           //don't increment ii
1653//                              endif
1654                               
1655//                              xval = ~(dataval & ~(2^32 - 2^8)) & 127
1656//                              yval = ((dataval & ~(2^32 - 2^16 ))/2^8) & 127
1657//                              time_lsw =  (dataval & ~(2^32 - 2^29 ))/2^16            //this method gives a FP result!! likely since the "^" operation gives FP result...
1658
1659                                xval = xMax - (dataval & 255)                                           //last 8 bits (7-0)
1660                                yval = (dataval & 65280)/256                                            //bits 15-8, right shift by 2^8
1661
1662                                time_lsw = (dataval & 536805376)/65536                  //13 bits, 28-16, right shift by 2^16 (result is integer)
1663
1664                                if(verbose)
1665//                                      printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1666                                        printf "%u : %u : %u : %u\r",dataval,timeval,xval,yval
1667                                endif
1668                               
1669////                            if(keep==decFac)                        //don't reset keep yet, do this only when ii increments
1670                                        xLoc[ii] = xval
1671                                        yLoc[ii] = yval
1672////                            endif
1673                               
1674                                // don't fill in the time yet, or increment the index ii
1675                                // the next event MUST be ATMIR with the MSW time bits
1676                                //
1677//                              verbose = 0
1678                                break
1679                        case ATMIR:  // 1
1680                                if(verbose)
1681                                        printf "MIR : "
1682                                endif
1683
1684                                time_msw =  (dataval & 536805376)/65536                 //13 bits, 28-16, right shift by 2^16
1685                                timeval = trunc( nRoll*roll_time + (time_msw * (8192)) + time_lsw )
1686                                if (timeval > t_longest)
1687                                        t_longest = timeval
1688                                endif
1689                                if(verbose)
1690//                                      printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1691                                        printf "d=%u : t=%u : msw=%u : lsw=%u : tlong=%u\r",dataval,timeval,time_msw,time_lsw,t_longest
1692                                endif
1693                               
1694                                // the XY position was in the previous event ATXYM
1695////                            if(keep == decFac)
1696                                        timePt[ii] = timeval
1697////                            endif
1698
1699                                bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1700                                if(bit29 != 0)          // bit 29 set is a T0 event
1701                                        //Printf "bit29 = 1 at ii = %d : type = %d\r",ii,type
1702                                        T0Time[tmpT0] = timeval
1703                                        T0EventNum[tmpT0] = ii
1704                                        tmpT0 += 1
1705                                        // reset nRoll = 0 for calcluating the time
1706                                        nRoll = 0
1707                                endif
1708                               
1709////                            if(keep == decFac)                     
1710                                        ii+=1
1711////                                    keep = 0
1712////                            endif
1713//                              verbose = 0
1714                                break
1715                        case ATMAR:  // 3
1716                                if(verbose3)
1717//                                      verbose = 15
1718//                                      verbose = 2
1719                                        printf "MAR : "
1720                                endif
1721                               
1722                                // do something with the rollover event?
1723                               
1724                                // check bit 29
1725                                bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1726                                nRoll += 1
1727// not doing anything with these bits yet       
1728                                bit28 = (dataval & 0x10000000)/268435456                //bit 28 only, shift by 2^28   
1729                                bit27 = (dataval & 0x08000000)/134217728        //bit 27 only, shift by 2^27
1730
1731                                if(verbose3)
1732                                        printf "d=%u : b29=%u : b28=%u : b27=%u : #Roll=%u \r",dataval,bit29, bit28, bit27,nRoll
1733                                endif
1734                               
1735                                if(bit29 != 0)          // bit 29 set is a T0 event
1736                                        //Printf "bit29 = 1 at ii = %d : type = %d\r",ii,type
1737                                        T0Time[tmpT0] = timeval
1738                                        T0EventNum[tmpT0] = ii
1739                                        tmpT0 += 1
1740                                        // reset nRoll = 0 for calcluating the time
1741                                        nRoll = 0
1742                                endif
1743                               
1744                                rolloverHappened = 1
1745
1746                                break
1747                endswitch
1748               
1749//              if(ii<18)
1750//                      printf "TYPE=%d : ii=%d : d=%u : t=%u : msw=%u : lsw=%u : %u : %u \r",type,ii,dataval,timeval,time_msw,time_lsw,xval,yval
1751//              endif   
1752                       
1753        while(1)
1754       
1755        Close fileref
1756       
1757        printf("Igor full file read done in  ")
1758        toc()
1759       
1760        Print "Events removed (Igor) = ",numRemoved
1761       
1762        sPrintf tmpStr,"\rBad Rollover Events = %d (%4.4g %% of events)",numBad,numBad/numXYevents*100
1763        dispStr += tmpStr
1764        sPrintf tmpStr,"\rTotal Events Removed = %d (%4.4g %% of events)",numRemoved,numRemoved/numXYevents*100
1765        dispStr += tmpStr
1766        SetDataFolder root:
1767       
1768        return(0)
1769       
1770End
1771
1772//////////////
1773//
1774// This calls the XOP, as an operation to load the events
1775//
1776// -- it's about 35x faster than the Igor code, so I guess that's OK.
1777//
1778// conditional compile the whole inner workings in case XOP is not present
1779Function LoadEvents_XOP()
1780#if (exists("EventLoadWave")==4)
1781       
1782//      NVAR time_msw = root:Packages:NIST:Event:gEvent_time_msw
1783//      NVAR time_lsw = root:Packages:NIST:Event:gEvent_time_lsw
1784        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1785       
1786        SVAR filepathstr = root:Packages:NIST:Event:gEvent_logfile
1787        SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
1788       
1789        SetDataFolder root:Packages:NIST:Event
1790
1791
1792
1793        Variable fileref
1794        String buffer
1795        String fileStr,tmpStr
1796        Variable dataval,timeval,type,numLines,verbose,verbose3
1797        Variable xval,yval,rollBit,nRoll,roll_time,bit29,bit28,bit27
1798        Variable ii,flaggedEvent,rolloverHappened,numBad=0,tmpPP=0,tmpT0=0
1799        Variable Xmax, yMax
1800       
1801        xMax = 127              // number the detector from 0->127
1802        yMax = 127
1803       
1804        numLines = 0
1805
1806        //Have to declare local variables for Loadwave so that this compiles without XOP.
1807        String S_waveNames
1808        //  and those for the XOP
1809        Variable V_nXYevents,V_num1,V_num2,V_num3,V_num0,V_totBytes,V_numPP,V_numT0,V_numDL,V_numFF,V_numZero
1810        Variable V_numBad,V_numRemoved
1811       
1812        // what I really need is the number of XY events
1813        Variable numXYevents,num1,num2,num3,num0,totBytes,numPP,numT0,numDL,numFF,numZero
1814        Variable numRemoved
1815        numXYevents = 0
1816        num0 = 0
1817        num1 = 0
1818        num2 = 0
1819        num3 = 0
1820        numPP = 0
1821        numT0 = 0
1822        numDL = 0
1823        numFF = 0
1824        numZero = 0
1825        numRemoved = 0
1826
1827// get the total number of bytes in the file
1828        Open/R fileref as filepathstr
1829                FStatus fileref
1830        Close fileref
1831
1832        totBytes = V_logEOF
1833        Print "total bytes = ", totBytes
1834       
1835        if(EventDataType(filePathStr) != 1)             //if not hst extension of raw data
1836                Abort "Data must be raw event data with file extension .hst for this operation"
1837                return(0)
1838        endif
1839//
1840//      Print "scan only"
1841//      tic()
1842//              EventLoadWave/R/N=EventWave/W filepathstr
1843//      toc()
1844
1845////
1846//
1847//  use the XOP operation to load in the data
1848// -- this does everything - the pre-scan and creating the waves
1849//
1850// need to zero the waves before loading, just in case
1851//
1852
1853        NVAR removeBadEvents = root:Packages:NIST:Event:gRemoveBadEvents
1854
1855tic()
1856
1857//      Wave/Z wave0=wave0
1858//      Wave/Z wave1=wave1
1859//      Wave/Z wave2=wave2
1860//
1861//      if(WaveExists(wave0))
1862//              MultiThread wave0=0
1863//      endif
1864//      if(WaveExists(wave1))
1865//              MultiThread wave1=0
1866//      endif
1867//      if(WaveExists(wave2))
1868//              MultiThread wave2=0
1869//      endif
1870
1871        if(removeBadEvents)
1872                EventLoadWave/R/N=EventWave filepathstr
1873        else
1874                EventLoadWave/N=EventWave  filepathstr
1875        endif
1876
1877
1878        Print "XOP files loaded = ",S_waveNames
1879
1880////            -- copy the waves over to xLoc,yLoc,timePt
1881        Wave/Z EventWave0=EventWave0
1882        Wave/Z EventWave1=EventWave1
1883        Wave/Z EventWave2=EventWave2
1884       
1885       
1886        Duplicate/O EventWave0,xLoc
1887        KillWaves/Z EventWave0
1888
1889        Duplicate/O EventWave1,yLoc
1890        KillWaves/Z EventWave1
1891
1892        Duplicate/O EventWave2,timePt
1893        KillWaves/Z EventWave2
1894
1895// could do this, but rescaled time will neeed to be converted to SP (or DP)
1896// and Igor loader was written with Make generating SP/DP waves
1897        // /I/U is unsigned 32-bit integer (for the time)
1898        // /B/U is unsigned 8-bit integer (max val=255) for the x and y values
1899       
1900//      Redimension/B/U xLoc,yLoc
1901//      Redimension/I/U timePt
1902
1903        // access the variables from the XOP
1904        numT0 = V_numT0
1905        numPP = V_numPP
1906        num0 = V_num0
1907        num1 = V_num1
1908        num2 = V_num2
1909        num3 = V_num3
1910        numXYevents = V_nXYevents
1911        numZero = V_numZero
1912        numBad = V_numBad
1913        numRemoved = V_numRemoved
1914       
1915        Print "(XOP) numT0 = ",numT0   
1916        Print "num0 = ",num0   
1917        Print "num1 = ",num1   
1918        Print "num2 = ",num2   
1919        Print "num3 = ",num3   
1920       
1921
1922// dispStr will be displayed on the panel
1923        fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
1924       
1925        sprintf tmpStr, "%s: %d total bytes\r",fileStr,totBytes
1926        dispStr = tmpStr
1927        sprintf tmpStr,"numXYevents = %d\r",numXYevents
1928        dispStr += tmpStr
1929        sprintf tmpStr,"PP = %d  :  ",numPP
1930        dispStr += tmpStr
1931        sprintf tmpStr,"ZeroData = %d\r",numZero
1932        dispStr += tmpStr
1933        sprintf tmpStr,"Rollover = %d",num3
1934        dispStr += tmpStr
1935
1936        toc()
1937       
1938        Print "Events removed (XOP) = ",numRemoved
1939       
1940        sPrintf tmpStr,"\rBad Rollover Events = %d (%4.4g %% of events)",numBad,numBad/numXYevents*100
1941        dispStr += tmpStr
1942        sPrintf tmpStr,"\rTotal Events Removed = %d (%4.4g %% of events)",numRemoved,numRemoved/numXYevents*100
1943        dispStr += tmpStr
1944
1945
1946// simply to compile a table of # XY vs # bytes
1947//      Wave/Z nxy = root:numberXY
1948//      Wave/Z nBytes = root:numberBytes
1949//      if(WaveExists(nxy) && WaveExists(nBytes))
1950//              InsertPoints 0, 1, nxy,nBytes
1951//              nxy[0] = numXYevents
1952//              nBytes[0] = totBytes
1953//      endif
1954
1955        SetDataFolder root:
1956
1957#endif 
1958        return(0)
1959       
1960End
1961
1962//////////////
1963
1964Proc BinEventBarGraph()
1965       
1966        DoWindow/F EventBarGraph
1967        if(V_flag == 0)
1968                PauseUpdate; Silent 1           // building window...
1969                String fldrSav0= GetDataFolder(1)
1970                SetDataFolder root:Packages:NIST:Event:
1971                Display /W=(110,705,610,1132)/N=EventBarGraph /K=1 binCount vs binEndTime
1972                SetDataFolder fldrSav0
1973                ModifyGraph mode=5
1974                ModifyGraph marker=19
1975                ModifyGraph lSize=2
1976                ModifyGraph rgb=(0,0,0)
1977                ModifyGraph msize=2
1978                ModifyGraph hbFill=2
1979                ModifyGraph gaps=0
1980                ModifyGraph usePlusRGB=1
1981                ModifyGraph toMode=1
1982                ModifyGraph useBarStrokeRGB=1
1983                ModifyGraph standoff=0
1984                SetAxis left 0,*
1985                Label bottom "\\Z14Time (seconds)"
1986                Label left "\\Z14Number of Events"
1987        endif
1988End
1989
1990
1991Proc ShowBinTable()
1992
1993        DoWindow/F BinEventTable
1994        if(V_flag == 0)
1995                PauseUpdate; Silent 1           // building window...
1996                String fldrSav0= GetDataFolder(1)
1997                SetDataFolder root:Packages:NIST:Event:
1998                Edit/W=(498,699,1003,955) /K=1/N=BinEventTable binCount,binEndTime,timeWidth
1999                ModifyTable format(Point)=1,sigDigits(binEndTime)=8,width(binEndTime)=100
2000                SetDataFolder fldrSav0
2001        endif
2002EndMacro
2003
2004
2005// only show the first 1500 data points
2006//
2007Proc ShowRescaledTimeGraph()
2008
2009        DoWindow/F RescaledTimeGraph
2010        if(V_flag == 0)
2011                PauseUpdate; Silent 1           // building window...
2012                String fldrSav0= GetDataFolder(1)
2013                SetDataFolder root:Packages:NIST:Event:
2014                Display /W=(25,44,486,356)/K=1/N=RescaledTimeGraph rescaledTime
2015                SetDataFolder fldrSav0
2016                ModifyGraph mode=0
2017                ModifyGraph marker=19
2018                ModifyGraph rgb(rescaledTime)=(0,0,0)
2019                ModifyGraph msize=1
2020//              SetAxis/A=2 left                        //only autoscale the visible data (based on the bottom limits)
2021                SetAxis bottom 0,1500
2022                ErrorBars rescaledTime OFF
2023                Label left "\\Z14Time (seconds)"
2024                Label bottom "\\Z14Event number"
2025                ShowInfo
2026        endif
2027       
2028EndMacro
2029
2030
2031
2032Proc ExportSlicesAsVAX(firstNum,prefix)
2033        Variable firstNum=1
2034        String prefix="SAMPL"
2035
2036        SaveSlicesAsVAX(firstNum,prefix[0,4])           //make sure that the prefix is 5 chars
2037End
2038
2039//////// procedures to be able to export the slices as RAW VAX files.
2040//
2041// 1- load the raw data file to use the header (it must already be in RAW)
2042// 1.5- copy the raw data to the temp folder (STO)
2043// 1.7- ask for the prefix and starting run number (these are passed in)
2044// 2- copy the slice of data to the temp folder (STO)
2045// 3- touch up the time/counts in the slice header values in STO
2046// 4- write out the VAX file
2047// 5- repeat (2-4) for the number of slices
2048//
2049//
2050Function SaveSlicesAsVAX(firstNum,prefix)
2051        Variable firstNum
2052        String prefix
2053
2054        DoAlert 1,"Is the full data file loaded as a RAW data file? If not, load it and start over..."
2055        if(V_flag == 2)
2056                return (0)
2057        endif
2058       
2059// copy the contents of RAW to STO so I can work from there
2060        CopyWorkContents("RAW","STO")
2061
2062        // now declare all of the waves, now that they are sure to be there
2063
2064        WAVE slicedData=root:Packages:NIST:Event:slicedData
2065        Make/O/D/N=(128,128) curSlice
2066       
2067        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
2068        WAVE binEndTime = root:Packages:NIST:Event:binEndTime
2069
2070        Wave rw=root:Packages:NIST:STO:realsRead
2071        Wave iw=root:Packages:NIST:STO:integersRead
2072        Wave/T tw=root:Packages:NIST:STO:textRead
2073        Wave data=root:Packages:NIST:STO:data
2074        Wave linear_data=root:Packages:NIST:STO:linear_data
2075       
2076       
2077        Wave rw_raw=root:Packages:NIST:RAW:realsRead
2078        Wave iw_raw=root:Packages:NIST:RAW:integersRead
2079        Wave/T tw_raw=root:Packages:NIST:RAW:textRead
2080
2081// for generating the alphanumeric
2082        String timeStr= secs2date(datetime,-1)
2083        String monthStr=StringFromList(1, timeStr  ,"/")
2084        String numStr="",labelStr
2085
2086        Variable ii,err,binFraction
2087       
2088        for(ii=0;ii<nslices;ii+=1)
2089
2090                //get the current slice and put it in the STO folder
2091                curSlice = slicedData[p][q][ii]
2092                data = curSlice
2093                linear_data = curSlice
2094               
2095                // touch up the header as needed
2096                // count time = iw[2]
2097                // monCt = rw[0]
2098                // detCt = rw[2]
2099                //tw[0] must now be the file name
2100                //
2101                // count time = fraction of total binning * total count time
2102                binFraction = (binEndTime[ii+1]-binEndTime[ii])/(binEndTime[nslices]-binEndTime[0])
2103               
2104                iw[2] = trunc(binFraction*iw_raw[2])
2105                rw[0] = trunc(binFraction*rw_raw[0])
2106                rw[2] = sum(curSlice,-inf,inf)          //total counts in slice
2107       
2108                if(firstNum<10)
2109                        numStr = "00"+num2str(firstNum)
2110                else
2111                        if(firstNum<100)
2112                                numStr = "0"+num2str(firstNum)
2113                        else
2114                                numStr = num2str(firstNum)
2115                        Endif
2116                Endif   
2117                tw[0] = prefix+numstr+".SA2_EVE_"+(num2char(str2num(monthStr)+64))+numStr
2118                labelStr = tw_raw[6]
2119               
2120                labelStr = PadString(labelStr,60,0x20)  //60 fortran-style spaces
2121                tw[6] = labelStr[0,59]
2122               
2123                //write out the file - this uses the tw[0] and home path
2124                Write_VAXRaw_Data("STO","",0)
2125
2126                //increment the run number, alpha
2127                firstNum += 1   
2128        endfor
2129
2130        return(0)
2131End
2132
2133
2134
2135
2136
2137/////////////
2138//The histogramming
2139//
2140// 6 AUG 2012
2141//
2142// from Igor Exchange, RGerkin
2143//  http://www.igorexchange.com/node/1373
2144// -- see the related thread on the mailing list
2145//
2146//Function Setup_JointHistogram()
2147//
2148////    tic()
2149//
2150//      make/D /o/n=1000000 data1=gnoise(1), data2=gnoise(1)
2151//      make/D /o/n=(25,25) myHist
2152//      setscale x,-3,3,myHist
2153//      setscale y,-3,3,myHist
2154//      IndexForHistogram(data1,data2,myhist)
2155//      Wave index=SavedIndex
2156//      EV_JointHistogram(data1,data2,myHist,index)
2157//      NewImage myHist
2158//     
2159////    toc()
2160//     
2161//End
2162
2163
2164Function EV_JointHistogram(w0,w1,hist,index)
2165        wave w0,w1,hist,index
2166 
2167        variable bins0=dimsize(hist,0)
2168        variable bins1=dimsize(hist,1)
2169        variable n=numpnts(w0)
2170        variable left0=dimoffset(hist,0)
2171        variable left1=dimoffset(hist,1)
2172        variable right0=left0+bins0*dimdelta(hist,0)
2173        variable right1=left1+bins1*dimdelta(hist,1)
2174       
2175        // Compute the histogram and redimension it. 
2176        histogram /b={0,1,bins0*bins1} index,hist
2177        redimension/D /n=(bins0,bins1) hist // Redimension to 2D. 
2178        setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension. 
2179        setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension. 
2180End
2181
2182
2183// histogram with a point range
2184//
2185// x- just need to send x2pnt or findLevel, or something similar to define the POINT
2186// values
2187//
2188// x- can also speed this up since the index only needs to be done once, so the
2189// histogram operation can be done separately, as the bins require
2190//
2191//
2192Function JointHistogramWithRange(w0,w1,hist,index,pt1,pt2)
2193        wave w0,w1,hist,index
2194        Variable pt1,pt2
2195 
2196        variable bins0=dimsize(hist,0)
2197        variable bins1=dimsize(hist,1)
2198        variable n=numpnts(w0)
2199        variable left0=dimoffset(hist,0)
2200        variable left1=dimoffset(hist,1)
2201        variable right0=left0+bins0*dimdelta(hist,0)
2202        variable right1=left1+bins1*dimdelta(hist,1)
2203
2204        // Compute the histogram and redimension it. 
2205        histogram /b={0,1,bins0*bins1}/R=[pt1,pt2] index,hist
2206        redimension/D /n=(bins0,bins1) hist // Redimension to 2D. 
2207        setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension. 
2208        setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension. 
2209End
2210
2211
2212// just does the indexing, creates wave SavedIndex in the current folder for the index
2213//
2214Function IndexForHistogram(w0,w1,hist)
2215        wave w0,w1,hist
2216 
2217        variable bins0=dimsize(hist,0)
2218        variable bins1=dimsize(hist,1)
2219        variable n=numpnts(w0)
2220        variable left0=dimoffset(hist,0)
2221        variable left1=dimoffset(hist,1)
2222        variable right0=left0+bins0*dimdelta(hist,0)
2223        variable right1=left1+bins1*dimdelta(hist,1)
2224 
2225        // Scale between 0 and the number of bins to create an index wave. 
2226        if(ThreadProcessorCount<4) // For older machines, matrixop is faster. 
2227                matrixop /free idx=round(bins0*(w0-left0)/(right0-left0))+bins0*round(bins1*(w1-left1)/(right1-left1))
2228        else // For newer machines with many cores, multithreading with make is faster. 
2229                make/free/n=(n) idx
2230                multithread idx=round(bins0*(w0-left0)/(right0-left0))+bins0*round(bins1*(w1-left1)/(right1-left1))
2231        endif
2232 
2233        KillWaves/Z SavedIndex
2234        MoveWave idx,SavedIndex
2235       
2236//      // Compute the histogram and redimension it. 
2237//      histogram /b={0,1,bins0*bins1} idx,hist
2238//      redimension /n=(bins0,bins1) hist // Redimension to 2D. 
2239//      setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension. 
2240//      setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension. 
2241End
2242
2243
2244
2245
2246
2247////////////// Post-processing of the event mode data
2248Proc ShowEventCorrectionPanel()
2249        DoWindow/F EventCorrectionPanel
2250        if(V_flag ==0)
2251                EventCorrectionPanel()
2252        EndIf
2253End
2254
2255Proc EventCorrectionPanel()
2256
2257        PauseUpdate; Silent 1           // building window...
2258        SetDataFolder root:Packages:NIST:Event:
2259       
2260        if(exists("rescaledTime") == 1)
2261                Display /W=(35,44,761,533)/K=2 rescaledTime
2262                DoWindow/C EventCorrectionPanel
2263                ModifyGraph mode=0
2264                ModifyGraph marker=19
2265                ModifyGraph rgb=(0,0,0)
2266                ModifyGraph msize=1
2267                ErrorBars rescaledTime OFF
2268                Label left "\\Z14Time (seconds)"
2269                Label bottom "\\Z14Event number"       
2270                SetAxis bottom 0,0.10*numpnts(rescaledTime)             //show 1st 10% of data for speed in displaying
2271               
2272                ControlBar 100
2273                Button button0,pos={18,12},size={70,20},proc=EC_AddCursorButtonProc,title="Cursors"
2274                Button button1,pos={18,38},size={80,20},proc=EC_ShowAllButtonProc,title="All Data"
2275                Button button2,pos={18,64},size={110,20},proc=EC_DoDifferential,title="Differential"
2276
2277
2278                SetVariable setvar0 pos={153,12},title="Tol",size={90,20},value=gStepTolerance
2279                Button button3,pos={153,38},size={100,20},proc=EC_AutoCorrectSteps,title="Auto Correct"
2280                Button button4,pos={153,64},size={110,20},proc=EC_AddFindNext,title="Add Find Next"
2281
2282               
2283                Button button5,pos={285,12},size={80,20},proc=EC_AddTimeButtonProc,title="Add 6.7s"
2284                Button button6,pos={285,38},size={80,20},proc=EC_SubtractTimeButtonProc,title="Subtr 6.7s"             
2285                Button button7,pos={285,64},size={90,20},proc=EC_TrimPointsButtonProc,title="Trim points"
2286               
2287       
2288                Button button8,pos={400,12},size={110,20},proc=EC_FindStepButton_down,title="Find Step Down"
2289                Button button9,pos={400,38},size={110,20},proc=EC_FindStepButton_up,title="Find Step Up"
2290                Button button10,pos={400,64},size={100,20},proc=EC_FindOutlierButton,title="Find Outlier"
2291                Button button11,pos={520,12},size={20,20},proc=EC_NudgeCursor,title=">"
2292               
2293
2294                Button button12,pos={683,12},size={30,20},proc=EC_HelpButtonProc,title="?"
2295                Button button13,pos={625,38},size={90,20},proc=EC_SaveWavesButtonProc,title="Save Waves"
2296                Button button14,pos={655,64},size={60,20},proc=EC_DoneButtonProc,title="Done"
2297       
2298
2299        else
2300                DoAlert 0, "Please load some event data, then you'll have something to edit."
2301        endif
2302       
2303        SetDataFolder root:
2304       
2305EndMacro
2306
2307
2308Function EC_AutoCorrectSteps(ba)
2309        STRUCT WMButtonAction &ba
2310
2311        switch( ba.eventCode )
2312                case 2: // mouse up
2313                        // click code here
2314                        AutoFix_Rollover_Steps()
2315                       
2316                        break
2317                case -1: // control being killed
2318                        break
2319        endswitch
2320               
2321        return(0)
2322End
2323
2324Function EC_NudgeCursor(ba)
2325        STRUCT WMButtonAction &ba
2326
2327        switch( ba.eventCode )
2328                case 2: // mouse up
2329                        // click code here
2330                        // nudge the leftmost cursor to the right two points
2331                        // presumably to bypass a desired time step
2332                        //
2333       
2334                        Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
2335                        Variable ptA,ptB,lo
2336                        ptA = pcsr(A)
2337                        ptB = pcsr(B)
2338                        lo=min(ptA,ptB)
2339
2340                        if(lo == ptA)
2341                                Cursor/P A rescaledTime ptA+2   //at the point + 2
2342                                Print "Nudged cursor A two points right"
2343                        else
2344                                Cursor/P B rescaledTime ptB+2   //at the point + 2
2345                                Print "Nudged cursor B two points right"
2346                        endif
2347                       
2348                        break
2349                case -1: // control being killed
2350                        break
2351        endswitch
2352               
2353        return(0)
2354End
2355
2356Function EC_AddFindNext(ba)
2357        STRUCT WMButtonAction &ba
2358
2359        switch( ba.eventCode )
2360                case 2: // mouse up
2361                        // click code here
2362                        // add time     
2363                        EC_AddTimeButtonProc(ba)
2364                       
2365                // re-do the differential
2366                        EC_DoDifferential("")
2367                       
2368                // find the next step down
2369                        EC_FindStepButton_down("")
2370                       
2371                        break
2372                case -1: // control being killed
2373                        break
2374        endswitch
2375               
2376        return(0)
2377End
2378
2379
2380
2381Function EC_AddCursorButtonProc(ba) : ButtonControl
2382        STRUCT WMButtonAction &ba
2383
2384        switch( ba.eventCode )
2385                case 2: // mouse up
2386                        // click code here
2387                        SetDataFolder root:Packages:NIST:Event:
2388                       
2389                        Wave rescaledTime = rescaledTime
2390                        Cursor/P A rescaledTime 0
2391                        Cursor/P B rescaledTime numpnts(rescaledTime)-1
2392                        ShowInfo
2393                        SetDataFolder root:
2394                        break
2395                case -1: // control being killed
2396                        break
2397        endswitch
2398
2399        return 0
2400End
2401
2402// updates the longest time (as does every operation of adjusting the data)
2403//
2404Function EC_AddTimeButtonProc(ba) : ButtonControl
2405        STRUCT WMButtonAction &ba
2406
2407        switch( ba.eventCode )
2408                case 2: // mouse up
2409                        // click code here
2410                        SetDataFolder root:Packages:NIST:Event:
2411                       
2412                        Wave rescaledTime = rescaledTime
2413                        Wave timePt = timePt
2414                        Variable rollTime,rollTicks,ptA,ptB,lo,hi
2415                       
2416                        rollTicks = 2^26                                // in ticks
2417                        rollTime = 2^26*1e-7            // in seconds
2418                        ptA = pcsr(A)
2419                        ptB = pcsr(B)
2420                        lo=min(ptA,ptB)
2421                        hi=max(ptA,ptB)
2422
2423                        MultiThread timePt[lo,hi] += rollTicks
2424                        MultiThread rescaledTime[lo,hi] += rollTime
2425
2426                        // updates the longest time (as does every operation of adjusting the data)
2427                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2428                        t_longest = waveMax(rescaledTime)
2429                       
2430                        SetDataFolder root:
2431                        break
2432                case -1: // control being killed
2433                        break
2434        endswitch
2435
2436        return 0
2437End
2438
2439Function EC_SubtractTimeButtonProc(ba) : ButtonControl
2440        STRUCT WMButtonAction &ba
2441
2442        switch( ba.eventCode )
2443                case 2: // mouse up
2444                        // click code here
2445                        SetDataFolder root:Packages:NIST:Event:
2446                       
2447                        Wave rescaledTime = rescaledTime
2448                        Wave timePt = timePt
2449                        Variable rollTime,rollTicks,ptA,ptB,lo,hi
2450                       
2451                        rollTicks = 2^26                                // in ticks
2452                        rollTime = 2^26*1e-7            // in seconds
2453                        ptA = pcsr(A)
2454                        ptB = pcsr(B)
2455                        lo=min(ptA,ptB)
2456                        hi=max(ptA,ptB)
2457                       
2458                        MultiThread timePt[lo,hi] -= rollTicks
2459                        MultiThread rescaledTime[lo,hi] -= rollTime
2460
2461                        // updates the longest time (as does every operation of adjusting the data)
2462                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2463                        t_longest = waveMax(rescaledTime)
2464                       
2465                        SetDataFolder root:
2466                       
2467                        break
2468                case -1: // control being killed
2469                        break
2470        endswitch
2471
2472        return 0
2473End
2474
2475// points removed are inclusive
2476//
2477// put both cursors on the same point to remove just that single point
2478//
2479Function EC_TrimPointsButtonProc(ba) : ButtonControl
2480        STRUCT WMButtonAction &ba
2481
2482        switch( ba.eventCode )
2483                case 2: // mouse up
2484                        // click code here
2485                        SetDataFolder root:Packages:NIST:Event:
2486                       
2487                        Wave rescaledTime = rescaledTime
2488                        Wave timePt = timePt
2489                        Wave xLoc = xLoc
2490                        Wave yLoc = yLoc
2491                        Variable rollTime,ptA,ptB,numElements,lo,hi
2492                       
2493                        rollTime = 2^26*1e-7            // in seconds
2494                        ptA = pcsr(A)
2495                        ptB = pcsr(B)
2496                        lo=min(ptA,ptB)
2497                        hi=max(ptA,ptB)                 
2498                        numElements = abs(ptA-ptB)+1                    //so points removed are inclusive
2499                        DeletePoints lo, numElements, rescaledTime,timePt,xLoc,yLoc
2500                       
2501                        printf "Points %g to %g have been deleted in rescaledTime, timePt, xLoc, and yLoc\r",ptA,ptB
2502                       
2503                        // updates the longest time (as does every operation of adjusting the data)
2504                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2505                        t_longest = waveMax(rescaledTime)
2506                       
2507                        SetDataFolder root:
2508                       
2509                        break
2510                case -1: // control being killed
2511                        break
2512        endswitch
2513
2514        return 0
2515End
2516
2517// un-sort the data first, then save it
2518Function EC_SaveWavesButtonProc(ba) : ButtonControl
2519        STRUCT WMButtonAction &ba
2520
2521        switch( ba.eventCode )
2522                case 2: // mouse up
2523                        // click code here
2524                       
2525//                      Execute "UndoTheSorting()"
2526                       
2527                        SetDataFolder root:Packages:NIST:Event:
2528                       
2529                        Wave rescaledTime = rescaledTime
2530                        Wave timePt = timePt
2531                        Wave xLoc = xLoc
2532                        Wave yLoc = yLoc
2533                        Save/T xLoc,yLoc,timePt,rescaledTime            //will ask for a name
2534                       
2535                        SetDataFolder root:
2536                        break
2537                case -1: // control being killed
2538                        break
2539        endswitch
2540
2541        return 0
2542End
2543
2544//
2545// this duplicates all of the bits that would be done if the "load" button was pressed
2546//
2547Function EC_ImportWavesButtonProc(ba) : ButtonControl
2548        STRUCT WMButtonAction &ba
2549
2550        switch( ba.eventCode )
2551                case 2: // mouse up
2552                        // click code here
2553                        SetDataFolder root:Packages:NIST:Event:
2554
2555                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2556                        SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
2557                        String tmpStr="",fileStr,filePathStr
2558                       
2559                        // load in the waves, saved as Igor text to preserve the data type
2560                        PathInfo/S catPathName          //should set the next dialog to the proper path...
2561                        filePathStr = PromptForPath("Pick the edited event data")
2562                        if(EventDataType(filePathStr) != 2)             //if not itx extension of edited data
2563                                Abort "Data must be edited data with file extension .itx for this operation"
2564                                return(0)
2565                        endif
2566                        LoadWave/T/O/P=catPathName filePathStr
2567//                      filePathStr = S_fileName
2568
2569                        if(strlen(S_fileName) == 0)
2570                                //user cancelled
2571                                DoAlert 0,"No file selected, nothing done."
2572                                return(0)
2573                        endif
2574                       
2575                        NVAR mode = root:Packages:NIST:Event:gEvent_Mode                                // ==0 for "stream", ==1 for Oscillatory
2576                        // clear out the old sort index, if present, since new data is being loaded
2577                        KillWaves/Z OscSortIndex
2578                        Wave timePt=timePt
2579                        Wave rescaledTime=rescaledTime
2580                       
2581                        t_longest = waveMax(rescaledTime)               //should be the last point
2582                       
2583       
2584                        fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
2585                        sprintf tmpStr, "%s: a user-modified event file\r",fileStr
2586                        dispStr = tmpStr
2587       
2588                        SetDataFolder root:
2589                        break
2590                case -1: // control being killed
2591                        break
2592        endswitch
2593
2594        return 0
2595End
2596
2597
2598Function EC_ShowAllButtonProc(ba) : ButtonControl
2599        STRUCT WMButtonAction &ba
2600
2601        switch( ba.eventCode )
2602                case 2: // mouse up
2603                        // click code here
2604                        SetAxis/A
2605                        break
2606                case -1: // control being killed
2607                        break
2608        endswitch
2609
2610        return 0
2611End
2612
2613Function EC_HelpButtonProc(ba) : ButtonControl
2614        STRUCT WMButtonAction &ba
2615
2616        switch( ba.eventCode )
2617                case 2: // mouse up
2618                        // click code here
2619                        DisplayHelpTopic/Z "Event Mode Data[Correcting for things that go wrong]"
2620                        break
2621                case -1: // control being killed
2622                        break
2623        endswitch
2624
2625        return 0
2626End
2627
2628Function EC_DoneButtonProc(ba) : ButtonControl
2629        STRUCT WMButtonAction &ba
2630
2631        switch( ba.eventCode )
2632                case 2: // mouse up
2633                        // click code here
2634                        DoWindow/K EventCorrectionPanel
2635                        break
2636                case -1: // control being killed
2637                        break
2638        endswitch
2639
2640        return 0
2641End
2642
2643//upDown 5 or -5 looks for spikes +5 or -5 std deviations from mean
2644//
2645// will search from the leftmost cursor to the end. this allows skipping of oscillations
2646// that are not timing errors. It may introduce other issues, but we'll see what happens
2647//
2648Function PutCursorsAtStep(upDown)
2649        Variable upDown
2650       
2651        SetDataFolder root:Packages:NIST:Event:
2652
2653        Wave rescaledTime=rescaledTime
2654        Wave rescaledTime_DIF=rescaledTime_DIF
2655        Variable avg,pt,zoom
2656        Variable ptA,ptB,startPt
2657               
2658        zoom = 200              //points in each direction
2659       
2660        WaveStats/M=1/Q rescaledTime_DIF
2661        avg = V_avg
2662       
2663        ptA = pcsr(A)
2664        ptB = pcsr(B)
2665        startPt = min(ptA,ptB)
2666       
2667        FindLevel/P/Q/R=[startPt] rescaledTime_DIF abs(avg)*upDown              // in case the average is negative
2668        if(V_flag==0)
2669                pt = V_levelX
2670                WaveStats/Q/R=[pt-zoom,pt+zoom] rescaledTime            // find the max/min y-values within the point range
2671        else
2672                Print "Level not found"
2673//              return(0)
2674        endif
2675       
2676        Variable loLeft,hiLeft, loBottom,hiBottom
2677        loLeft = V_min*0.98             //+/- 2%
2678        hiLeft = V_max*1.02
2679       
2680        SetAxis left loLeft,hiLeft
2681        SetAxis bottom pnt2x(rescaledTime,pt-zoom),pnt2x(rescaledTime,pt+zoom)
2682       
2683        Cursor/P A rescaledTime pt+2    //at the point
2684        Cursor/P B rescaledTime numpnts(rescaledTime)-1         //at the end
2685
2686        SetDataFolder root:
2687
2688        return(0)
2689End
2690
2691
2692// find the max (or min) of the rescaled time set
2693// and place both cursors there
2694Function fFindOutlier()
2695
2696        SetDataFolder root:Packages:NIST:Event:
2697
2698        Wave rescaledTime=rescaledTime
2699        Variable avg,pt,zoom,maxPt,minPt,maxVal,minVal
2700       
2701        zoom = 200              //points in each direction
2702       
2703        WaveStats/M=1/Q rescaledTime
2704        maxPt = V_maxLoc
2705        minPt = V_minLoc
2706        avg = V_avg
2707        maxVal = abs(V_max)
2708        minVal = abs(V_min)
2709
2710        pt = abs(maxVal - avg) > abs(minVal - avg) ? maxPt : minPt
2711       
2712//      Variable loLeft,hiLeft, loBottom,hiBottom
2713//      loLeft = V_min*0.98             //+/- 2%
2714//      hiLeft = V_max*1.02
2715       
2716//      SetAxis left loLeft,hiLeft
2717//      SetAxis bottom pnt2x(rescaledTime,pt-zoom),pnt2x(rescaledTime,pt+zoom)
2718       
2719        Cursor/P A rescaledTime pt              //at the point
2720        Cursor/P B rescaledTime pt              //at the same point
2721
2722        SetDataFolder root:
2723       
2724        return(0)
2725End
2726
2727Function EC_FindStepButton_down(ctrlName) : ButtonControl
2728        String ctrlName
2729       
2730//      Variable upDown = -5
2731        NVAR upDown = root:Packages:NIST:Event:gStepTolerance
2732       
2733        PutCursorsAtBigStep(upDown)
2734//      PutCursorsAtStep(-1*upDown)
2735
2736        return(0)
2737end
2738
2739
2740Function EC_FindStepButton_up(ctrlName) : ButtonControl
2741        String ctrlName
2742       
2743//      Variable upDown = 5
2744        NVAR upDown = root:Packages:NIST:Event:gStepTolerance
2745
2746        PutCursorsAtBigStep(upDown)
2747//      PutCursorsAtStep(upDown)
2748
2749        return(0)
2750end
2751
2752// if the Trim button section is uncommented, it's "Zap outlier"
2753//
2754Function EC_FindOutlierButton(ctrlName) : ButtonControl
2755        String ctrlName
2756       
2757        fFindOutlier()
2758//
2759//      STRUCT WMButtonAction ba
2760//      ba.eventCode = 2
2761//
2762//      EC_TrimPointsButtonProc(ba)
2763
2764        return(0)
2765end
2766
2767Function EC_DoDifferential(ctrlName) : ButtonControl
2768        String ctrlName
2769       
2770        DifferentiatedTime()
2771        DoWindow/F EventCorrectionPanel
2772       
2773        //if trace is not on graph, add it
2774        SetDataFolder root:Packages:NIST:Event:
2775
2776        String list = WaveList("*_DIF", ";", "WIN:EventCorrectionPanel")
2777        if(strlen(list) == 0)
2778                AppendToGraph/R rescaledTime_DIF
2779                ModifyGraph msize=1,rgb(rescaledTime_DIF)=(65535,0,0)
2780                ReorderTraces rescaledTime,{rescaledTime_DIF}           // put the differential behind the event data
2781        endif
2782        SetDataFolder root:
2783        return(0)
2784end
2785
2786//////////////   Custom Bins  /////////////////////
2787//
2788//
2789//
2790// make sure that the bins are defined and the waves exist before
2791// trying to draw the panel
2792//
2793Proc Show_CustomBinPanel()
2794        DoWindow/F CustomBinPanel
2795        if(V_flag ==0)
2796                Init_CustomBins()
2797                CustomBinPanel()
2798        EndIf
2799End
2800
2801
2802Function Init_CustomBins()
2803
2804        NVAR nSlice = root:Packages:NIST:Event:gEvent_nslices
2805        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2806
2807        Variable/G root:Packages:NIST:Event:gEvent_ForceTmaxBin=1               //==1 to enforce t_longest in user-defined custom bins
2808
2809        SetDataFolder root:Packages:NIST:Event:
2810               
2811        Make/O/D/N=(nSlice) timeWidth
2812        Make/O/D/N=(nSlice+1) binEndTime,binCount
2813       
2814        timeWidth = t_longest/nslice
2815        binEndTime = p
2816        binCount = p+1 
2817       
2818        SetDataFolder root:
2819       
2820        return(0)
2821End
2822
2823////////////////       
2824//
2825// Allow custom definitions of the bin widths
2826//
2827// Define by the number of bins, and the time width of each bin
2828//
2829// This shares the number of slices and the maximum time with the main panel
2830//
2831Proc CustomBinPanel()
2832        PauseUpdate; Silent 1           // building window...
2833        NewPanel /W=(130,44,851,455)/K=2 /N=CustomBinPanel
2834        DoWindow/C CustomBinPanel
2835        ModifyPanel fixedSize=1//,noEdit =1
2836        SetDrawLayer UserBack
2837       
2838        Button button0,pos={654,42}, size={50,20},title="Done",fSize=12
2839        Button button0,proc=CB_Done_Proc
2840        Button button1,pos={663,14},size={40,20},proc=CB_HelpButtonProc,title="?"
2841        Button button2,pos={216,42},size={80,20},title="Update",proc=CB_UpdateWavesButton       
2842        SetVariable setvar1,pos={23,13},size={160,20},title="Number of slices",fSize=12
2843        SetVariable setvar1,proc=CB_NumSlicesSetVarProc,value=root:Packages:NIST:Event:gEvent_nslices
2844        SetVariable setvar2,pos={24,44},size={160,20},title="Max Time (s)",fSize=10
2845        SetVariable setvar2,value=root:Packages:NIST:Event:gEvent_t_longest     
2846
2847        CheckBox chkbox1,pos={216,14},title="Enforce Max Time?"
2848        CheckBox chkbox1,variable = root:Packages:NIST:Event:gEvent_ForceTmaxBin
2849        Button button3,pos={500,14},size={90,20},proc=CB_SaveBinsButtonProc,title="Save Bins"
2850        Button button4,pos={500,42},size={100,20},proc=CB_ImportBinsButtonProc,title="Import Bins"     
2851               
2852        SetDataFolder root:Packages:NIST:Event:
2853
2854        Display/W=(291,86,706,395)/HOST=CustomBinPanel/N=BarGraph binCount vs binEndTime
2855        ModifyGraph mode=5
2856        ModifyGraph marker=19
2857        ModifyGraph lSize=2
2858        ModifyGraph rgb=(0,0,0)
2859        ModifyGraph msize=2
2860        ModifyGraph hbFill=2
2861        ModifyGraph gaps=0
2862        ModifyGraph usePlusRGB=1
2863        ModifyGraph toMode=1
2864        ModifyGraph useBarStrokeRGB=1
2865        ModifyGraph standoff=0
2866        SetAxis left 0,*
2867        Label bottom "\\Z14Time (seconds)"
2868        Label left "\\Z14Number of Events"
2869        SetActiveSubwindow ##
2870       
2871        // and the table
2872        Edit/W=(13,87,280,394)/HOST=CustomBinPanel/N=T0
2873        AppendToTable/W=CustomBinPanel#T0 timeWidth,binEndTime
2874        ModifyTable width(Point)=40
2875        SetActiveSubwindow ##
2876       
2877        SetDataFolder root:
2878       
2879EndMacro
2880
2881// save the bins - use Igor Text format
2882//
2883Function CB_SaveBinsButtonProc(ba) : ButtonControl
2884        STRUCT WMButtonAction &ba
2885
2886        switch( ba.eventCode )
2887                case 2: // mouse up
2888                        // click code here
2889
2890                        SetDataFolder root:Packages:NIST:Event:
2891
2892                        Wave timeWidth = timeWidth
2893                        Wave binEndTime = binEndTime
2894                       
2895                        Save/T timeWidth,binEndTime                     //will ask for a name
2896
2897                        break
2898                case -1: // control being killed
2899                        break
2900        endswitch
2901
2902        SetDataFolder root:
2903       
2904        return 0
2905End
2906
2907// Import the bins - use Igor Text format
2908//
2909// -- be sure that the number of bins is reset
2910// -?- how about the t_longest? - this should be set by the load, not here
2911//
2912// -- loads in timeWidth and binEndTime
2913//
2914Function CB_ImportBinsButtonProc(ba) : ButtonControl
2915        STRUCT WMButtonAction &ba
2916
2917        switch( ba.eventCode )
2918                case 2: // mouse up
2919                        // click code here
2920                        NVAR nSlice = root:Packages:NIST:Event:gEvent_nslices
2921
2922                        SetDataFolder root:Packages:NIST:Event:
2923
2924                        // prompt for the load of data
2925                        LoadWave/T/O
2926                        if(strlen(S_fileName) == 0)
2927                                //user cancelled
2928                                DoAlert 0,"No file selected, nothing done."
2929                                return(0)
2930                        endif
2931
2932                        Wave timeWidth = timeWidth
2933                        nSlice = numpnts(timeWidth)
2934                       
2935                        break
2936                case -1: // control being killed
2937                        break
2938        endswitch
2939
2940        SetDataFolder root:
2941       
2942        return 0
2943End
2944
2945
2946
2947//
2948// can either use the widths as stated -- then the end time may not
2949// match the actual end time of the data set
2950//
2951// -- or --
2952//
2953// enforce the end time of the data set to be the end time of the bins,
2954// then the last bin width must be reset to force the constraint
2955//
2956//
2957Function CB_UpdateWavesButton(ba) : ButtonControl
2958        STRUCT WMButtonAction &ba
2959
2960        switch( ba.eventCode )
2961                case 2: // mouse up
2962                        // click code here
2963                        NVAR nSlice = root:Packages:NIST:Event:gEvent_nslices
2964                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2965                        NVAR enforceTmax = root:Packages:NIST:Event:gEvent_ForceTmaxBin
2966                       
2967                        // update the waves, and recalculate everything for the display
2968                        SetDataFolder root:Packages:NIST:Event:
2969
2970                        Wave timeWidth = timeWidth
2971                        Wave binEndTime = binEndTime
2972                        Wave binCount = binCount
2973                       
2974                        // use the widths as entered
2975                        binEndTime[0] = 0
2976                        binEndTime[1,] = binEndTime[p-1] + timeWidth[p-1]
2977                       
2978                        // enforce the longest time as the end bin time
2979                        // note that this changes the last time width
2980                        if(enforceTmax)
2981                                binEndTime[nSlice] = t_longest
2982                                timeWidth[nSlice-1] = t_longest - binEndTime[nSlice-1]
2983                        endif
2984                       
2985                        binCount = p+1
2986                        binCount[nSlice] = 0            // last point is zero, just for display
2987//                      binCount *= sign(timeWidth)             //to alert to negative time bins
2988                       
2989                        // make the timeWidth bold and red if the widths are negative
2990                        WaveStats/Q timeWidth
2991                        if(V_min < 0)
2992                                ModifyTable/W=CustomBinPanel#T0 style(timeWidth)=1,rgb(timeWidth)=(65535,0,0)                   
2993                        else
2994                                ModifyTable/W=CustomBinPanel#T0 style(timeWidth)=0,rgb(timeWidth)=(0,0,0)                       
2995                        endif
2996                       
2997                        break
2998                case -1: // control being killed
2999                        break
3000        endswitch
3001
3002        SetDataFolder root:
3003       
3004        return 0
3005End
3006
3007Function CB_HelpButtonProc(ba) : ButtonControl
3008        STRUCT WMButtonAction &ba
3009
3010        switch( ba.eventCode )
3011                case 2: // mouse up
3012                        // click code here
3013                        DisplayHelpTopic/Z "Event Mode Data[Setting up Custom Bin Widths]"
3014                        break
3015                case -1: // control being killed
3016                        break
3017        endswitch
3018
3019        return 0
3020End
3021
3022Function CB_Done_Proc(ba) : ButtonControl
3023        STRUCT WMButtonAction &ba
3024       
3025        String win = ba.win
3026        switch (ba.eventCode)
3027                case 2:
3028                        DoWindow/K CustomBinPanel
3029                        break
3030        endswitch
3031        return(0)
3032End
3033
3034
3035Function CB_NumSlicesSetVarProc(sva) : SetVariableControl
3036        STRUCT WMSetVariableAction &sva
3037
3038        switch( sva.eventCode )
3039                case 1: // mouse up
3040                case 2: // Enter key
3041                case 3: // Live update
3042                        Variable dval = sva.dval
3043                        String sval = sva.sval
3044                        SetDataFolder root:Packages:NIST:Event:
3045
3046                        Wave timeWidth = timeWidth
3047                        Wave binEndTime = binEndTime
3048                       
3049                        Redimension/N=(dval) timeWidth
3050                        Redimension/N=(dval+1) binEndTime,binCount
3051                       
3052                        SetDataFolder root:
3053                       
3054                        break
3055                case -1: // control being killed
3056                        break
3057        endswitch
3058
3059        return 0
3060End
3061
3062
3063///////////////////
3064//
3065// utility to split a large file
3066// 100 MB is the recommended size
3067// events can be clipped here, so be sure to trim the ends of the
3068// resulting files as needed.
3069//
3070// - works like the unix 'split' command
3071//
3072//
3073
3074Proc SplitBigFile(splitSize, baseStr)
3075        Variable splitSize = 100
3076        String baseStr="split"
3077        Prompt splitSize,"Target file size, in MB (< 150 MB!!)"
3078        Prompt baseStr,"File prefix, number will be appended"
3079       
3080        if(splitSize > root:Packages:NIST:Event:gEventFileTooLarge)
3081                Abort "File split must be less than 150 MB. Please try again"
3082        endif
3083       
3084        fSplitBigFile(splitSize, baseStr)
3085       
3086        ShowSplitFileTable()
3087End
3088
3089Function/S fSplitBigFile(splitSize, baseStr)
3090        Variable splitSize
3091        String baseStr         
3092
3093
3094        String fileName=""              // File name, partial path, full path or "" for dialog.
3095        Variable refNum
3096        String str
3097        SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3098       
3099        listStr=""              //initialize output list
3100
3101        Variable readSize=1e6           //1 MB
3102        Make/O/B/U/N=(readSize) aBlob                   //1MB worth
3103        Variable numSplit
3104        Variable num,ii,jj,outRef,frac
3105        String thePath, outStr
3106       
3107        Printf "SplitSize = %u MB\r",splitSize
3108        splitSize = trunc(splitSize) * 1e6              // now in bytes
3109       
3110       
3111        // Open file for read.
3112        Open/R/Z=2/F="????"/P=catPathName refNum as fileName
3113        thePath = ParseFilePath(1, S_fileName, ":", 1, 0)
3114        Print "thePath = ",thePath
3115       
3116        // Store results from Open in a safe place.
3117        Variable err = V_flag
3118        String fullPath = S_fileName
3119
3120        if (err == -1)
3121                Print "cancelled by user."
3122                return ("")
3123        endif
3124
3125        FStatus refNum
3126       
3127        Printf "total # bytes = %u\r",V_logEOF
3128
3129        numSplit=0
3130        if(V_logEOF > splitSize)
3131                numSplit = trunc(V_logEOF/splitSize)
3132        endif
3133
3134        frac = V_logEOF - numSplit*splitSize
3135        Print "numSplit = ",numSplit
3136        Printf "frac = %u\r",frac
3137       
3138        num=0
3139        if(frac > readSize)
3140                num = trunc(frac/readSize)
3141        endif
3142
3143       
3144        frac = frac - num*readSize
3145
3146        Print "num = ",num
3147        Printf "frac = %u\r",frac
3148       
3149//      baseStr = "split"
3150       
3151        for(ii=0;ii<numSplit;ii+=1)
3152                outStr = (thePath+baseStr+num2str(ii)+".hst")
3153//              Print "outStr = ",outStr
3154                Open outRef as outStr
3155
3156                for(jj=0;jj<(splitSize/readSize);jj+=1)
3157                        FBinRead refNum,aBlob
3158                        FBinWrite outRef,aBlob
3159                endfor
3160
3161                Close outRef
3162//              listStr += outStr+";"           //no, this is the FULL path
3163                listStr += baseStr+num2str(ii)+".hst"+";"
3164        endfor
3165
3166        Make/O/B/U/N=(frac) leftover
3167        // ii was already incremented past the loop
3168        outStr = (thePath+baseStr+num2str(ii)+".hst")
3169        Open outRef as outStr
3170        for(jj=0;jj<num;jj+=1)
3171                FBinRead refNum,aBlob
3172                FBinWrite outRef,aBlob
3173        endfor
3174        FBinRead refNum,leftover
3175        FBinWrite outRef,leftover
3176
3177        Close outRef
3178//      listStr += outStr+";"
3179        listStr += baseStr+num2str(ii)+".hst"+";"
3180
3181        FSetPos refNum,V_logEOF
3182        Close refNum
3183       
3184        KillWaves/Z aBlob,leftover
3185        return(listStr)
3186End
3187
3188// allows the list of loaded files to be edited
3189Function ShowSplitFileTable()
3190
3191        SVAR str = root:Packages:NIST:Event:gSplitFileList
3192       
3193        WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3194        if(waveExists(tw) != 1)
3195                Make/O/T/N=1 root:Packages:NIST:Event:SplitFileWave
3196                WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3197        endif
3198
3199        List2TextWave(str,tw)
3200        Edit tw
3201
3202        return(0)
3203End
3204
3205
3206//// save the sliced data, and accumulate slices
3207//  *** this works with sliced data -- that is data that has been PROCESSED
3208//
3209// need some way of ensuring that the slices match up since I'm blindly adding them together.
3210//
3211// mode = 0             wipe out the old accumulated, copy slicedData to accumulatedData
3212// mode = 1             add current slicedData to accumulatedData
3213// mode = 2             copy accumulatedData to slicedData in preparation of export or display
3214// mode = 3             unused...
3215//
3216//      "Split Large File",SplitBigFile()
3217//      "Accumulate First Slice",AccumulateSlices(0)
3218//      "Add Current Slice",AccumulateSlices(1)
3219//      "Display Accumulated Slices",AccumulateSlices(2)       
3220//
3221Function AccumulateSlicesButton(ctrlName) : ButtonControl
3222        String ctrlName
3223       
3224        Variable mode
3225        mode = str2num(ctrlName[strlen(ctrlName)-1])
3226//      Print "mode=",mode
3227        AccumulateSlices(mode)
3228       
3229        return(0)
3230End
3231
3232// added an extra (optional) parameter for silent operation - to be used when a list of
3233// files are processed - so the user can walk away as it works. Using the buttons from the
3234// panel will still present the alerts
3235//
3236Function AccumulateSlices(mode,[skipAlert])
3237        Variable mode,skipAlert
3238       
3239        if(ParamIsDefault(skipAlert))
3240                skipAlert = 0
3241        endif
3242        SetDataFolder root:Packages:NIST:Event:
3243
3244        switch(mode)   
3245                case 0:
3246                        if(!skipAlert)
3247                                DoAlert 0,"The current data has been copied to the accumulated set. You are now ready to add more data."
3248                        endif
3249                        KillWaves/Z accumulatedData
3250                        Duplicate/O slicedData accumulatedData 
3251                        KillWaves/Z accum_BinCount              // the accumulated BinCount
3252                        Duplicate/O binCount accum_binCount     
3253                        break
3254                case 1:
3255                        if(!skipAlert)
3256                                DoAlert 0,"The current data has been added to the accumulated data. You can add more data."
3257                        endif
3258                        Wave acc=accumulatedData
3259                        Wave cur=slicedData
3260                        acc += cur
3261                        Wave binCount = binCount
3262                        Wave accum_binCount = accum_BinCount
3263                        accum_BinCount += binCount
3264                        binCount = accum_binCount                       // !! NOTE - the bin count wave is now the total
3265//                      AccumulateSlices(2,skipAlert=skipAlert)                                         // and a recursive call to show the total!
3266                        break
3267                case 2:
3268                        if(!skipAlert)
3269                                DoAlert 0,"The accumulated data is now the display data and is ready for display or export."
3270                        endif
3271                        Duplicate/O accumulatedData slicedData
3272                        // do something to "touch" the display to force it to update
3273                        NVAR gLog = root:Packages:NIST:Event:gEvent_logint
3274                        LogIntEvent_Proc("",gLog)
3275                        break
3276                default:                       
3277                               
3278        endswitch
3279
3280        SetDataFolder root:
3281        return(0)
3282end
3283
3284
3285////////////////////////////////////////////
3286//
3287// Panel and procedures for decimation
3288//
3289////////////////////////////////////////////
3290
3291//Function E_ShowDecimateButton(ctrlName) : ButtonControl
3292//      String ctrlName
3293//
3294//      DoWindow/F DecimatePanel
3295//      if(V_flag ==0)
3296//              Execute "DecimatePanel()"
3297//      endif
3298//End
3299//
3300//
3301//Proc DecimatePanel() //: Panel
3302//     
3303//      PauseUpdate; Silent 1           // building window...
3304//      NewPanel /W=(1602,44,1961,380)/K=1
3305////    ShowTools/A
3306//      Button button0,pos={29,15},size={100,20},proc=SplitFileButtonProc,title="Split Big File"
3307//      SetVariable setvar0,pos={182,55},size={150,15},title="Decimation factor",fsize=10
3308//      SetVariable setvar0,limits={1,inf,1},value= root:Packages:NIST:Event:gDecimation
3309//      Button button1,pos={26,245},size={150,20},proc=LoadDecimateButtonProc,title="Load and Decimate"
3310//      Button button2,pos={25,277},size={150,20},proc=ConcatenateButtonProc,title="Concatenate"
3311//      Button button3,pos={25,305},size={150,20},proc=DisplayConcatenatedButtonProc,title="Display Concatenated"
3312//      Button button4,pos={29,52},size={130,20},proc=Stream_LoadDecim,title="Load From List"
3313//     
3314//      GroupBox group0 title="Manual Controls",size={185,112},pos={14,220}
3315//EndMacro
3316
3317
3318Function SplitFileButtonProc(ctrlName) : ButtonControl
3319        String ctrlName
3320
3321        Execute "SplitBigFile()"
3322End
3323
3324
3325// show all of the data
3326//
3327Proc ShowDecimatedGraph()
3328
3329        DoWindow/F DecimatedGraph
3330        if(V_flag == 0)
3331                PauseUpdate; Silent 1           // building window...
3332                String fldrSav0= GetDataFolder(1)
3333                SetDataFolder root:Packages:NIST:Event:
3334                Display /W=(25,44,486,356)/K=1/N=DecimatedGraph rescaledTime_dec
3335                SetDataFolder fldrSav0
3336                ModifyGraph mode=4
3337                ModifyGraph marker=19
3338                ModifyGraph rgb(rescaledTime_dec)=(0,0,0)
3339                ModifyGraph msize=1
3340                ErrorBars rescaledTime_dec OFF
3341                Label left "\\Z14Time (seconds)"
3342                Label bottom "\\Z14Event number"
3343                ShowInfo
3344        endif
3345       
3346EndMacro
3347
3348// data has NOT been processed
3349//
3350// so work with x,y,t, and rescaled time
3351// variables -- t_longest
3352Function ConcatenateButtonProc(ctrlName) : ButtonControl
3353        String ctrlName
3354       
3355        DoAlert 1,"Is this the first file?"
3356        Variable first = V_flag
3357       
3358        fConcatenateButton(first)
3359       
3360        return(0)
3361End
3362
3363Function fConcatenateButton(first)
3364        Variable first
3365
3366
3367        SetDataFolder root:Packages:NIST:Event:
3368
3369        Wave timePt_dTmp=timePt_dTmp
3370        Wave xLoc_dTmp=xLoc_dTmp
3371        Wave yLoc_dTmp=yLoc_dTmp
3372        Wave rescaledTime_dTmp=rescaledTime_dTmp
3373       
3374        NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3375        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3376       
3377       
3378        if(first==1)            //1==yes, 2==no
3379                //then copy the files over, adjusting the time to start from zero
3380                // rescaledTime starts from zero (set by the loader)
3381
3382                timePt_dTmp -= timePt_dTmp[0]                   //subtract the first value
3383               
3384                Duplicate/O timePt_dTmp timePt_dec
3385                Duplicate/O xLoc_dTmp xLoc_dec
3386                Duplicate/O yLoc_dTmp yLoc_dec
3387                Duplicate/O rescaledTime_dTmp rescaledTime_dec
3388               
3389                t_longest_dec = t_longest
3390       
3391        else
3392                // concatenate the files + adjust the time
3393                Wave timePt_dec=timePt_dec
3394                Wave xLoc_dec=xLoc_dec
3395                Wave yLoc_dec=yLoc_dec
3396                Wave rescaledTime_dec=rescaledTime_dec
3397
3398                // adjust the times -- assuming they add
3399                // rescaledTime starts from zero (set by the loader)
3400                //
3401                //
3402                rescaledTime_dTmp += rescaledTime_dec[numpnts(rescaledTime_dec)-1]
3403                rescaledTime_dTmp += abs(rescaledTime_dec[numpnts(rescaledTime_dec)-1] - rescaledTime_dec[numpnts(rescaledTime_dec)-2])
3404               
3405                timePt_dTmp -= timePt_dTmp[0]                   //subtract the first value     
3406               
3407                timePt_dTmp += timePt_dec[numpnts(timePt_dec)-1]                // offset by the last point
3408                timePt_dTmp += abs(timePt_dec[numpnts(timePt_dec)-1] - timePt_dec[numpnts(timePt_dec)-2])               // plus delta so there's not a flat step
3409               
3410                Concatenate/NP/O {timePt_dec,timePt_dTmp}, tmp
3411                Duplicate/O tmp timePt_dec
3412               
3413                Concatenate/NP/O {xLoc_dec,xLoc_dTmp}, tmp
3414                Duplicate/O tmp xLoc_dec
3415               
3416                Concatenate/NP/O {yLoc_dec,yLoc_dTmp}, tmp
3417                Duplicate/O tmp yLoc_dec
3418               
3419                Concatenate/NP/O {rescaledTime_dec,rescaledTime_dTmp}, tmp
3420                Duplicate/O tmp rescaledTime_dec
3421               
3422
3423                KillWaves tmp
3424
3425                t_longest_dec = rescaledTime_dec[numpnts(rescaledTime_dec)-1]
3426
3427        endif
3428       
3429       
3430        SetDataFolder root:
3431       
3432        return(0)
3433
3434End
3435
3436Function DisplayConcatenatedButtonProc(ctrlName) : ButtonControl
3437        String ctrlName
3438
3439        //copy the files over to the display set for processing
3440        SetDataFolder root:Packages:NIST:Event:
3441
3442        Wave timePt_dec=timePt_dec
3443        Wave xLoc_dec=xLoc_dec
3444        Wave yLoc_dec=yLoc_dec
3445        Wave rescaledTime_dec=rescaledTime_dec
3446               
3447        NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3448        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3449       
3450        Duplicate/O timePt_dec timePt
3451        Duplicate/O xLoc_dec xLoc
3452        Duplicate/O yLoc_dec yLoc
3453        Duplicate/O rescaledTime_dec rescaledTime
3454       
3455        t_longest = t_longest_dec       
3456       
3457        SetDataFolder root:
3458       
3459        return(0)
3460
3461End
3462
3463
3464Proc DisplayForSlicing()
3465
3466        // plot the EventBarGraph?
3467                SetDataFolder root:Packages:NIST:Event:
3468                Display /W=(110,705,610,1132)/N=SliceGraph /K=1 binCount vs binEndTime
3469                ModifyGraph mode=5
3470                ModifyGraph marker=19
3471                ModifyGraph lSize=2
3472                ModifyGraph rgb=(0,0,0)
3473                ModifyGraph msize=2
3474                ModifyGraph hbFill=2
3475                ModifyGraph gaps=0
3476                ModifyGraph usePlusRGB=1
3477                ModifyGraph toMode=1
3478                ModifyGraph useBarStrokeRGB=1
3479                ModifyGraph standoff=0
3480
3481                SetDataFolder root:
3482
3483// append the differential
3484        AppendToGraph/R root:Packages:NIST:Event:rescaledTime_DIF vs root:Packages:NIST:Event:rescaledTime
3485        ModifyGraph rgb(rescaledTime_DIF)=(1,16019,65535)
3486
3487End
3488
3489
3490
3491// unused, old testing procedure
3492Function LoadDecimateButtonProc(ctrlName) : ButtonControl
3493        String ctrlName
3494
3495        LoadEventLog_Button("")
3496       
3497        // now decimate
3498        SetDataFolder root:Packages:NIST:Event:
3499
3500        Wave timePt=timePt
3501        Wave xLoc=xLoc
3502        Wave yLoc=yLoc
3503        NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3504
3505        NVAR decimation = root:Packages:NIST:Event:gDecimation
3506
3507
3508        Duplicate/O timePt, timePt_dTmp
3509        Duplicate/O xLoc, xLoc_dTmp
3510        Duplicate/O yLoc, yLoc_dTmp
3511        Resample/DOWN=(decimation)/N=1 timePt_dTmp
3512        Resample/DOWN=(decimation)/N=1 xLoc_dTmp
3513        Resample/DOWN=(decimation)/N=1 yLoc_dTmp
3514
3515
3516        Duplicate/O timePt_dTmp rescaledTime_dTmp
3517        rescaledTime_dTmp = 1e-7*(timePt_dTmp - timePt_dTmp[0])         //convert to seconds and start from zero
3518        t_longest_dec = waveMax(rescaledTime_dTmp)              //should be the last point
3519
3520        SetDataFolder root:
3521
3522       
3523End
3524
3525
3526
3527
3528
3529
3530
3531//
3532// loads a list of files, decimating each chunk as it is read in
3533//
3534Function Stream_LoadDecim(ctrlName)
3535        String ctrlName
3536       
3537        Variable fileref
3538        Variable num,ii
3539       
3540        SVAR filename = root:Packages:NIST:Event:gEvent_logfile
3541        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3542
3543        SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3544        NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3545        NVAR decimation = root:Packages:NIST:Event:gDecimation
3546
3547        String pathStr
3548        PathInfo catPathName
3549        pathStr = S_Path
3550
3551// if "stream" mode is not checked - abort
3552        NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
3553        if(gEventModeRadioVal != MODE_STREAM)
3554                Abort "The mode must be 'Stream' to use this function"
3555                return(0)
3556        endif
3557
3558// if the list has been edited, turn it into a list
3559        WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3560        if(WaveExists(tw))
3561                listStr = TextWave2SemiList(tw)
3562        else
3563                ShowSplitFileTable()
3564                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3565                return(0)
3566        endif
3567       
3568
3569        //loop through everything in the list
3570        num = ItemsInList(listStr)
3571        if(num == 0)
3572                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3573                return (0)
3574        endif
3575       
3576        for(ii=0;ii<num;ii+=1)
3577
3578// (1) load the file, prepending the path               
3579                filename = pathStr + StringFromList(ii, listStr  ,";")
3580                if(EventDataType(filename) != 1)                //if not hst extension of raw data
3581                        Abort "Data must be raw event data with file extension .hst for this operation"
3582                        return(0)
3583                endif           
3584
3585#if (exists("EventLoadWave")==4)
3586                LoadEvents_XOP()
3587#else
3588                LoadEvents_New_noXOP()
3589//              LoadEvents()
3590#endif 
3591
3592                SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root:
3593
3594                Wave timePt=timePt
3595                Wave xLoc=xLoc
3596                Wave yLoc=yLoc
3597                CleanupTimes(xLoc,yLoc,timePt)          //remove zeroes
3598
3599                Duplicate/O timePt rescaledTime
3600                MultiThread rescaledTime = 1e-7*(timePt-timePt[0])              //convert to seconds and start from zero
3601                t_longest = waveMax(rescaledTime)               //should be the last point
3602               
3603// (2) do the decimation, just on timePt. create rescaledTime from the decimated timePt
3604               
3605                Duplicate/O timePt, timePt_dTmp
3606                Duplicate/O xLoc, xLoc_dTmp
3607                Duplicate/O yLoc, yLoc_dTmp
3608                Resample/DOWN=(decimation)/N=1 timePt_dTmp
3609                Resample/DOWN=(decimation)/N=1 xLoc_dTmp
3610                Resample/DOWN=(decimation)/N=1 yLoc_dTmp
3611       
3612       
3613                Duplicate/O timePt_dTmp rescaledTime_dTmp
3614                rescaledTime_dTmp = 1e-7*(timePt_dTmp - timePt_dTmp[0])         //convert to seconds and start from zero
3615                t_longest_dec = waveMax(rescaledTime_dTmp)              //should be the last point
3616               
3617
3618// (3) concatenate the files + adjust the time
3619                fConcatenateButton(ii+1)                //passes 1 for the first time, >1 each other time
3620       
3621        endfor
3622
3623////            Now that everything is decimated and concatenated, create the rescaled time wave
3624//      SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root:
3625//      Wave timePt_dec = timePt_dec
3626//      Duplicate/O timePt_dec rescaledTime_dec
3627//      rescaledTime_dec = 1e-7*(timePt_dec - timePt_dec[0])            //convert to seconds and start from zero
3628//      t_longest_dec = waveMax(rescaledTime_dec)               //should be the last point
3629       
3630        DisplayConcatenatedButtonProc("")
3631       
3632        SetDataFolder root:
3633
3634        return(0)
3635End
3636
3637Function ShowList_ToLoad(ctrlName)
3638        String ctrlName
3639       
3640        ShowSplitFileTable()
3641       
3642        return(0)
3643End
3644
3645
3646//
3647// loads a list of files that have been adjusted and saved
3648// -- does not decimate
3649//
3650Function Stream_LoadAdjustedList(ctrlName)
3651        String ctrlName
3652       
3653        Variable fileref
3654
3655        SVAR filename = root:Packages:NIST:Event:gEvent_logfile
3656        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3657
3658        SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3659        NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3660//      NVAR decimation = root:Packages:NIST:Event:gDecimation
3661
3662        String pathStr
3663        PathInfo catPathName
3664        pathStr = S_Path
3665
3666// if "stream" mode is not checked - abort
3667        NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
3668        if(gEventModeRadioVal != MODE_STREAM)
3669                Abort "The mode must be 'Stream' to use this function"
3670                return(0)
3671        endif
3672
3673// if the list has been edited, turn it into a list
3674        WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3675        if(WaveExists(tw))
3676                listStr = TextWave2SemiList(tw)
3677        else
3678                ShowSplitFileTable()
3679                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3680                return(0)
3681        endif
3682       
3683
3684        //loop through everything in the list
3685        Variable num,ii
3686        num = ItemsInList(listStr)
3687       
3688        if(num == 0)
3689                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3690                return(0)
3691        endif
3692       
3693        for(ii=0;ii<num;ii+=1)
3694
3695// (1) load the file, prepending the path               
3696                filename = pathStr + StringFromList(ii, listStr  ,";")
3697                if(EventDataType(filename) != 2)                //if not itx extension of edited data
3698                        Abort "Data must be edited data with file extension .itx for this operation"
3699                        return(0)
3700                endif           
3701                SetDataFolder root:Packages:NIST:Event:
3702                LoadWave/T/O fileName
3703
3704                SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root: ??
3705
3706// this is what is loaded -- _dec extension is what is concatenated, and will be copied back later
3707                Wave timePt=timePt
3708                Wave xLoc=xLoc
3709                Wave yLoc=yLoc
3710                Wave rescaledTime=rescaledTime
3711
3712//              CleanupTimes(xLoc,yLoc,timePt)          //remove zeroes
3713
3714//              Duplicate/O timePt rescaledTime
3715//              rescaledTime = 1e-7*(timePt-timePt[0])          //convert to seconds and start from zero
3716//              t_longest = waveMax(rescaledTime)               //should be the last point
3717               
3718// (2) No decimation
3719               
3720                Duplicate/O timePt, timePt_dTmp
3721                Duplicate/O xLoc, xLoc_dTmp
3722                Duplicate/O yLoc, yLoc_dTmp
3723                Duplicate/O rescaledTime, rescaledTime_dTmp
3724
3725
3726// (3) concatenate the files + adjust the time
3727                fConcatenateButton(ii+1)                //passes 1 for the first time, >1 each other time
3728       
3729        endfor
3730       
3731        DisplayConcatenatedButtonProc("")               // this resets the longest time, too
3732               
3733        SetDataFolder root:
3734
3735        return(0)
3736End
3737
3738//
3739// loads a list of files that have been adjusted and saved
3740// -- does not decimate
3741// -- bins as they are loaded
3742//
3743Function Stream_LoadAdjList_BinOnFly(ctrlName)
3744        String ctrlName
3745       
3746        Variable fileref
3747
3748        SVAR filename = root:Packages:NIST:Event:gEvent_logfile
3749        SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3750
3751        String pathStr
3752        PathInfo catPathName
3753        pathStr = S_Path
3754
3755// if "stream" mode is not checked - abort
3756        NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
3757        if(gEventModeRadioVal != MODE_STREAM)
3758                Abort "The mode must be 'Stream' to use this function"
3759                return(0)
3760        endif
3761
3762// if the list has been edited, turn it into a list
3763        WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3764        if(WaveExists(tw))
3765                listStr = TextWave2SemiList(tw)
3766        else
3767                ShowSplitFileTable()
3768                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3769                return(0)
3770        endif
3771       
3772
3773        //loop through everything in the list
3774        Variable num,ii,jj
3775        num = ItemsInList(listStr)
3776        if(num == 0)
3777                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3778                return(0)
3779        endif
3780       
3781////////////////
3782// declarations for the binning
3783// for the first pass, do all of this
3784        Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
3785       
3786        Wave binnedData = root:Packages:NIST:Event:binnedData
3787        Wave xLoc = root:Packages:NIST:Event:xLoc
3788        Wave yLoc = root:Packages:NIST:Event:yLoc
3789
3790// now with the number of slices and max time, process the events
3791
3792        NVAR yesSortStream = root:Packages:NIST:Event:gSortStreamEvents         //do I sort the events?
3793        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3794        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
3795//      NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3796        NVAR t_segment_start = root:Packages:NIST:Event:gEvent_t_segment_start
3797        NVAR t_segment_end = root:Packages:NIST:Event:gEvent_t_segment_end
3798               
3799        SetDataFolder root:Packages:NIST:Event          //don't count on the folder remaining here
3800       
3801        Make/D/O/N=(128,128,nslices) slicedData
3802       
3803        Wave slicedData = slicedData
3804        Wave rescaledTime = rescaledTime
3805        Make/O/D/N=(128,128) tmpData
3806        Make/O/D/N=(nslices+1) binEndTime,binCount//,binStartTime
3807        Make/O/D/N=(nslices) timeWidth
3808        Wave binEndTime = binEndTime
3809        Wave timeWidth = timeWidth
3810        Wave binCount = binCount
3811
3812        slicedData = 0
3813        binCount = 0
3814
3815        variable p1,p2,t1,t2
3816
3817        t_segment_start = 0
3818// start w/file 1??     
3819        for(jj=0;jj<num;jj+=1)
3820
3821// (1) load the file, prepending the path               
3822                filename = pathStr + StringFromList(jj, listStr  ,";")
3823                if(EventDataType(filename) != 2)                //if not itx extension of edited data
3824                        Abort "Data must be edited data with file extension .itx for this operation"
3825                        return(0)
3826                endif           
3827                SetDataFolder root:Packages:NIST:Event:
3828                LoadWave/T/O fileName
3829
3830                SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root: ??
3831
3832// this is what is loaded --
3833                Wave timePt=timePt
3834                Wave xLoc=xLoc
3835                Wave yLoc=yLoc
3836                Wave rescaledTime=rescaledTime
3837
3838                if(jj==0)
3839                        timePt -= timePt[0]                     //make sure the first point is zero
3840                        t_segment_start = 0
3841                        t_segment_end = rescaledTime[numpnts(rescaledTime)-1]
3842                else
3843                        t_segment_start = t_segment_end
3844                        t_segment_end += rescaledTime[numpnts(rescaledTime)-1]
3845                       
3846                        rescaledTime += t_segment_start         
3847                endif
3848               
3849                Printf "jj=%g\t\tt_segment_start=%g\tt_segment_end=%g\r",jj,t_segment_start,t_segment_end
3850               
3851// this is the binning--
3852
3853//              del = t_longest/nslices
3854       
3855        // TODO
3856        // the global exists for this switch, but it is not implemented - not sure whether
3857        // it's correct to implement this at all --
3858        //
3859                if(yesSortStream == 1)
3860                        SortTimeData()
3861                endif
3862               
3863        // index the events before binning
3864        // if there is a sort of these events, I need to re-index the events for the histogram
3865        //      SetDataFolder root:Packages:NIST:Event
3866                IndexForHistogram(xLoc,yLoc,binnedData)
3867        //      SetDataFolder root:
3868                Wave index = root:Packages:NIST:Event:SavedIndex                //the index for the histogram
3869               
3870                for(ii=0;ii<nslices;ii+=1)
3871                        if(ii==0)
3872        //                      t1 = ii*del
3873        //                      t2 = (ii+1)*del
3874                                p1 = BinarySearch(rescaledTime,0)
3875                                p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
3876                        else
3877        //                      t2 = (ii+1)*del
3878                                p1 = p2+1               //one more than the old one
3879                                p2 = BinarySearch(rescaledTime,binEndTime[ii+1])               
3880                        endif
3881       
3882                        if(p1 == -1)
3883                                Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
3884                                p1 = 0          //set to the first point if it's off the end
3885                        Endif
3886                        if(p2 == -2)
3887                                Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
3888                                p2 = numpnts(rescaledTime)-1            //set to the last point if it's off the end
3889                        Endif
3890        //              Print p1,p2
3891       
3892                        tmpData=0
3893                        JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
3894        //              slicedData[][][ii] = tmpData[p][q]
3895                        slicedData[][][ii] += tmpData[p][q]
3896                       
3897//                      Duplicate/O tmpData,$("tmpData_"+num2str(ii)+"_"+num2str(jj))
3898                       
3899        //              binEndTime[ii+1] = t2
3900        //              binCount[ii] = sum(tmpData,-inf,inf)
3901                        binCount[ii] += sum(tmpData,-inf,inf)
3902                endfor
3903       
3904        endfor  //end loop over the file list
3905
3906// after all of the files have been loaded
3907        t_longest = t_segment_end               //this is the total time
3908
3909        Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
3910        Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
3911        logslicedData = log(slicedData)
3912       
3913               
3914        SetDataFolder root:
3915
3916        return(0)
3917End
3918
3919
3920/////////////////////////////////////
3921
3922// dd-mon-yyyy hh:mm:ss -> Event file name
3923// the VAX uses 24 hr time for hh
3924//
3925// scans as string elements since I'm reconstructing a string name
3926Function/S DateAndTime2HSTName(dateandtime)
3927        string dateAndTime
3928       
3929        String day,yr,hh,mm,ss,time_secs
3930        Variable mon
3931        string str,monStr,fileStr
3932       
3933        str=dateandtime
3934        sscanf str,"%2s-%3s-%4s %2s:%2s:%2s",day,monStr,yr,hh,mm,ss
3935        mon = monStr2num(monStr)
3936
3937        fileStr = "Event"+yr+num2str(mon)+day+hh+mm+ss+".hst"
3938        Print fileStr
3939
3940        return(fileStr)
3941end
3942
3943// dd-mon-yyyy hh:mm:ss -> Event file name
3944// the VAX uses 24 hr time for hh
3945//
3946// scans as string elements since I'm reconstructing a string name
3947Function DateAndTime2HSTNumber(dateandtime)
3948        string dateAndTime
3949       
3950        String day,yr,hh,mm,ss,time_secs
3951        Variable mon,num
3952        string str,monStr,fileStr
3953       
3954        str=dateandtime
3955        sscanf str,"%2s-%3s-%4s %2s:%2s:%2s",day,monStr,yr,hh,mm,ss
3956        mon = monStr2num(monStr)
3957
3958        fileStr = yr+num2str(mon)+day+hh+mm+ss
3959        num = str2num(fileStr)
3960
3961        return(num)
3962end
3963
3964Function HSTName2Num(str)
3965        String str
3966       
3967        Variable num
3968        sscanf str,"Event%d.hst",num
3969        return(num)
3970end
3971/////////////////////////////
3972
3973Proc GetListofITXorSplitFiles(searchStr)
3974        String searchStr = ".itx"
3975        ShowList_ToLoad("")
3976        GetListofITXHST(searchStr)
3977end
3978
3979// needs SANS_Utilites.ipf
3980Function GetListofITXHST(searchStr)
3981        String searchStr
3982       
3983        String fullList = IndexedFile(catpathName, -1, "????" )
3984       
3985        // list is currently everything -- trim it
3986        searchStr = "*" + searchStr + "*"
3987        String list = ListMatch(fullList, searchStr )
3988       
3989        Print "searchString = ",searchStr
3990        //
3991        Wave/T tw = root:Packages:NIST:Event:SplitFileWave
3992        List2TextWave(list,tw)
3993        Sort/A tw,tw                    //alphanumeric sort
3994        Edit tw
3995        return(0)
3996End
3997
3998//
3999// loads a list of files that have been adjusted and saved
4000// -- MUST run one through first manually to set the bin sapcing, then this
4001//    won't touch that, just process all of the files
4002// -- bins as they are loaded
4003//
4004Function Osc_LoadAdjList_BinOnFly(ctrlName)
4005        String ctrlName
4006       
4007        Variable fileref
4008
4009        SVAR filename = root:Packages:NIST:Event:gEvent_logfile
4010        SVAR listStr = root:Packages:NIST:Event:gSplitFileList
4011
4012        String pathStr,fileStr,tmpStr
4013        PathInfo catPathName
4014        pathStr = S_Path
4015
4016// if "Osc" mode is not checked - abort
4017        NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
4018        if(gEventModeRadioVal != MODE_OSCILL)
4019                Abort "The mode must be 'Oscillatory' to use this function"
4020                return(0)
4021        endif
4022
4023// if the list has been edited, turn it into a list
4024        WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
4025        if(WaveExists(tw))
4026                listStr = TextWave2SemiList(tw)
4027        else
4028                ShowSplitFileTable()
4029                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
4030                return(0)
4031        endif
4032       
4033
4034        //loop through everything in the list
4035        Variable num,ii,jj
4036        num = ItemsInList(listStr)
4037
4038//handle the first file differently
4039//(1) load (import edited)
4040        jj=0
4041        filename = pathStr + StringFromList(jj, listStr  ,";")
4042        if(EventDataType(filename) != 2)                //if not itx extension of edited data
4043                Abort "Data must be edited data with file extension .itx for this operation"
4044                return(0)
4045        endif
4046       
4047        SetDataFolder root:Packages:NIST:Event:
4048        LoadWave/T/O fileName
4049        SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root: ??
4050
4051        // clear out the old sort index, if present, since new data is being loaded
4052        KillWaves/Z OscSortIndex       
4053        SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
4054        sprintf tmpStr, "%s: a user-modified event file\r",StringFromList(jj, listStr  ,";")
4055        dispStr = tmpStr       
4056       
4057//(2) bin
4058        Osc_ProcessEventLog("")
4059       
4060//(3) display bin details
4061        Execute "ShowBinTable()"
4062        Execute "BinEventBarGraph()"
4063       
4064//(4) add first
4065        AccumulateSlices(0,skipAlert=1)         //bypass the dialog
4066
4067        // with remaining files
4068        // load
4069        // bin
4070        // add next
4071
4072// start w/file 1, since "0th" file was already processed
4073        for(jj=1;jj<num;jj+=1)
4074       
4075        //(1) load (import edited)
4076                filename = pathStr + StringFromList(jj, listStr  ,";")
4077                if(EventDataType(filename) != 2)                //if not itx extension of edited data
4078                        Abort "Data must be edited data with file extension .itx for this operation"
4079                        return(0)
4080                endif
4081                SetDataFolder root:Packages:NIST:Event:
4082                LoadWave/T/O fileName
4083                SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root: ??
4084       
4085                // clear out the old sort index, if present, since new data is being loaded
4086                KillWaves/Z OscSortIndex       
4087                SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
4088                sprintf tmpStr, "%s: a user-modified event file\r",StringFromList(jj, listStr  ,";")
4089                dispStr = tmpStr       
4090               
4091        //(2) bin
4092                Osc_ProcessEventLog("")
4093               
4094        //(3) display bin details
4095                Execute "ShowBinTable()"
4096                Execute "BinEventBarGraph()"
4097               
4098        //(4) add next
4099                AccumulateSlices(1,skipAlert=1)
4100       
4101        endfor  //end loop over the file list
4102
4103       
4104        //display total
4105        AccumulateSlices(2,skipAlert=1)
4106
4107        // let user know we're done
4108        DoAlert 0,"The list has been processed and is ready for export"
4109        SetDataFolder root:
4110
4111        return(0)
4112End
4113
4114
4115//returns 0 for unknown data
4116// 1 for raw hst files
4117// 2 for edited ITX files
4118//
4119//  TODO
4120//  -- currently, this blindly assumes an .hst extension for raw HST data, and .itx for edited data
4121//
4122Function EventDataType(filename)
4123        string filename
4124       
4125        Variable type
4126        Variable offset = strsearch(filename,".",Inf,1)         //work backwards
4127        String extension = filename[offset+1,strlen(filename)-1]
4128        //Print extension
4129        type = 0
4130        if(cmpstr(extension,"hst") == 0)
4131                type = 1
4132        endif
4133        if(cmpstr(extension,"itx") == 0)
4134                type = 2
4135        endif
4136        return(type)
4137end
4138
4139////////////
4140// this could be jazzed up quite a bit, but it is a first pass at
4141// figuring out what is going on with the events
4142//
4143Proc DecodeBinaryEvents(numToPrint)
4144        Variable numToPrint=10
4145       
4146        DecodeEvents(numToPrint)
4147end
4148
4149//      sscanf %x takes hex input and gives a real value output
4150// printf %d val gives the integer representation of the sscanf output
4151// printf %b val gives the binary representation.
4152Function DecodeEvents(numToPrint)
4153        Variable numToPrint
4154
4155        String filePathStr,buffer
4156        Variable fileRef,type,dataval,num0,num1,num2,num3
4157        Variable numT0,numPP,numZero,numXYevents,bit29
4158        Variable ii
4159
4160        ii=0
4161        Open/R fileref
4162        tic()
4163        do
4164                do
4165                        FReadLine fileref, buffer                       //skip the "blank" lines that have one character
4166                while(strlen(buffer) == 1)             
4167
4168                if (strlen(buffer) == 0)
4169                        break
4170                endif
4171               
4172                sscanf buffer,"%x",dataval
4173                if(ii<numToPrint)
4174                        printf "%s  %0.32b  ",buffer[0,7],dataval
4175                endif
4176               
4177                // two most sig bits (31-30)
4178                type = (dataval & 0xC0000000)/1073741824                //right shift by 2^30
4179
4180                if(ii<numToPrint)
4181                        printf "%0.2b  ",type
4182                endif
4183                                               
4184                if(type == 0)
4185                        num0 += 1
4186                        numXYevents += 1
4187                endif
4188                if(type == 2)
4189                        num2 += 1
4190                        numXYevents += 1
4191                endif
4192                if(type == 1)
4193                        num1 += 1
4194                endif
4195                if(type == 3)
4196                        num3 += 1
4197                endif   
4198               
4199                bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
4200                if(ii<numToPrint)
4201                        printf "%0.1b  ",bit29
4202                endif
4203                               
4204                if(type==0 || type==2)
4205                        numPP += round(bit29)
4206                endif
4207               
4208                if(type==1 || type==3)
4209                        numT0 += round(bit29)
4210                endif
4211               
4212                if(dataval == 0)
4213                        numZero += 1
4214                endif
4215               
4216                if(ii<numToPrint)
4217                        printf "\r"
4218                        ii += 1
4219                endif
4220               
4221        while(1)
4222        Close fileref
4223toc()
4224
4225        Print "(Igor) numT0 = ",numT0   
4226        Print "num0 = ",num0   
4227        Print "num1 = ",num1   
4228        Print "num2 = ",num2   
4229        Print "num3 = rollover = ",num3
4230        Print "numXY = ",numXYevents
4231        Print "numZero = ",numZero
4232        Print "numPP = ",numPP
4233       
4234        return(0)
4235
4236End
4237
4238
4239//////////////////
4240Function FixStepDown()
4241       
4242        SetDataFolder root:Packages:NIST:Event:
4243       
4244        Wave rescaledTime = rescaledTime
4245        Wave timePt = timePt
4246        Variable rollTime,rollTicks,ii,delta
4247       
4248        rollTicks = 2^26                                // in ticks
4249        rollTime = 2^26*1e-7            // in seconds
4250
4251        for(ii=0;ii<numpnts(rescaledTime)-1;ii+=1)
4252                delta = rescaledTime[ii+1] - rescaledTime[ii]
4253                if(delta < -6.6 && delta > -6.8)                //assume anything this large is a step down
4254                print ii, delta
4255                        MultiThread timePt[ii+1,] += rollTicks
4256                        MultiThread rescaledTime[ii+1,] += rollTime
4257               
4258                        // updates the longest time (as does every operation of adjusting the data)
4259                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
4260                        t_longest = waveMax(rescaledTime)
4261                endif
4262       
4263        endfor
4264
4265        SetDataFolder root:
4266
4267        return(0)
4268End
4269
4270Function FixStepUp()
4271       
4272        SetDataFolder root:Packages:NIST:Event:
4273       
4274        Wave rescaledTime = rescaledTime
4275        Wave timePt = timePt
4276        Variable rollTime,rollTicks,ii,delta
4277       
4278        rollTicks = 2^26                                // in ticks
4279        rollTime = 2^26*1e-7            // in seconds
4280
4281        for(ii=0;ii<numpnts(rescaledTime)-1;ii+=1)
4282                delta = rescaledTime[ii+1] - rescaledTime[ii]
4283                if(delta > 6.6 && delta < 6.8)          //assume anything this large is a step up
4284                print ii, delta
4285                        MultiThread timePt[ii+1,] -= rollTicks
4286                        MultiThread rescaledTime[ii+1,] -= rollTime
4287               
4288                        // updates the longest time (as does every operation of adjusting the data)
4289                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
4290                        t_longest = waveMax(rescaledTime)
4291                endif
4292       
4293        endfor
4294
4295        SetDataFolder root:
4296
4297        return(0)
4298End
4299
4300//    added a test function AutoFix_Rollover_Steps() that will search for steps (simple delta) that
4301//    are +/- 0.1s away from the 6.7s missed rollover. This function looks for up, down, then re-calculates
4302//    the derivative.
4303//  - this way, both the simple steps, and the "square" steps can both be corrected.
4304//
4305Function AutoFix_Rollover_Steps()
4306// fix steps up, then down (order doesn't matter)
4307        Print "Fixing steps of 6.7s up"
4308        FixStepUp()
4309        Print "Fixing steps of 6.7s down"
4310        FixStepDown()
4311//      re-do the differential
4312        DifferentiatedTime()
4313        return(0)
4314End
4315
4316
4317//upDown looks for a step up or down?
4318//
4319// plan is to be able to locate large (t0) steps from oscillations that are not timing errors
4320//
4321//
4322// will search from the leftmost cursor to the end. this allows skipping of oscillations
4323// that are not timing errors. It may introduce other issues, but we'll see what happens
4324//
4325Function PutCursorsAtBigStep(tol)
4326        Variable tol
4327       
4328        SetDataFolder root:Packages:NIST:Event:
4329
4330        Wave rescaledTime=rescaledTime
4331        Variable ii,delta,ptA,ptB,startPt,pt
4332               
4333        ptA = pcsr(A)
4334        ptB = pcsr(B)
4335        startPt = min(ptA,ptB)
4336               
4337        for(ii=startPt;ii<numpnts(rescaledTime)-1;ii+=1)
4338                delta = rescaledTime[ii+1] - rescaledTime[ii]           //if there is a step down, this will be negative
4339                if(abs(delta) > tol)
4340                        print ii, delta
4341                        pt = ii
4342                        break
4343                endif   
4344        endfor
4345       
4346        Cursor/P A rescaledTime pt+1    //at the point+1
4347        Cursor/P B rescaledTime numpnts(rescaledTime)-1         //at the end
4348
4349        SetDataFolder root:
4350
4351        return(0)
4352       
4353       
4354End
4355
4356
4357
4358Proc InsertTimeReset(period)
4359        Variable period
4360       
4361        fInsertTimeReset(period)
4362End
4363
4364
4365//
4366// for event data where a time reset signal was not sent, but the data is
4367// actually periodic. This function will reset the time at the input period.
4368// This assumes that you accurately know the period, and that the start time
4369// of the data coincides with the start time of the period, otherwise the
4370// time reset will happen in the middle of the cycle.
4371//
4372// period = period of reset (s)
4373//
4374Function fInsertTimeReset(period)
4375        Variable period
4376       
4377        SetDataFolder root:Packages:NIST:Event:
4378       
4379        Wave rescaledTime = rescaledTime
4380        Wave timePt = timePt
4381        Variable rollTime,rollTicks,ii,delta
4382       
4383        Variable period_ticks
4384       
4385        period_ticks = period*1e7               //period in ticks
4386       
4387
4388        for(ii=0;ii<numpnts(rescaledTime)-1;ii+=1)
4389                if(rescaledTime[ii] > period)
4390                        MultiThread timePt[ii,] -= period_ticks
4391                        MultiThread rescaledTime[ii,] -= period
4392                endif
4393        endfor
4394
4395// updates the longest time (as does every operation of adjusting the data)
4396        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
4397        t_longest = waveMax(rescaledTime)
4398        SetDataFolder root:
4399
4400        return(0)
4401End
4402
4403
4404/////////////////////////////////
4405//
4406//
4407//  Improved loading of SANS Event files
4408//      without the use of an XOP (in case macOS / Igor XOPs can't be loaded)
4409//
4410//      28 FEB 2020 SRK
4411//
4412
4413// TODO
4414// -- still need to clean this up to provide an identical entry point and functionality
4415//   to the existing non-XOP loader
4416// -- all routines need to be made DataFolder aware. currently everything is in root:
4417// -- all waves need to be explicitly declared. currently the assumption is that they exist, in root:
4418// -- be sure that global variables are declared and set with the loaded values (t_max, values for status)
4419// -- clean up (kill) anything not needed so that I can save memory
4420
4421
4422
4423//
4424//
4425// this now works properly and with reasonable speed:
4426// relative times:
4427// XOP = 1
4428// old Igor code = 28
4429// this new code = 6.3
4430//
4431// loaded variables, times, XY location have all been verified vs. XOP and old Igor routines
4432//
4433
4434// this code loads 563 MB in 114 s (as predicted by scaling up XOP load speed)
4435// loads 979 MB in 266 s (slower than predicted)
4436
4437
4438Function LoadEvents_New_noXOP()
4439
4440        SVAR fname = root:Packages:NIST:Event:gEvent_logfile
4441
4442        Variable numBad = 0
4443       
4444//      String fname = DoOpenFileDialog("select an event file")
4445
4446tic()           
4447        LoadEventAsHex(fname)
4448        Wave/T/Z newWave = root:Packages:NIST:Event:newWave
4449
4450        RemoveFFF(newWave)
4451       
4452        GetBitsFromEvents()
4453        Wave/Z bit29 = root:Packages:NIST:Event:bit29
4454        Wave/Z events = root:Packages:NIST:Event:events
4455        Wave/Z time_lsw = root:Packages:NIST:Event:time_lsw
4456        Wave/Z type = root:Packages:NIST:Event:type
4457        Wave/Z xloc = root:Packages:NIST:Event:xloc
4458        Wave/Z yloc = root:Packages:NIST:Event:yloc
4459
4460        numBad = CleanUpBeginning(type,events,bit29,xloc,yloc,time_lsw)
4461
4462        DecodeEvents_New(type,events,bit29,xloc,yloc,time_lsw,numBad)
4463        Wave/Z timePt = root:Packages:NIST:Event:timePt
4464        Wave/Z deletePtFlag = root:Packages:NIST:Event:deletePtFlag
4465
4466//
4467// see the trick for setting the deletePtFlag wave that I use in DecodeEvents_New
4468// -- I needed to do this since the sort operation does not preserve the order of
4469// any items with the same value --- from the help file:
4470// "The algorithm used does not maintain the relative position of items with the same key value."
4471//
4472//
4473        Sort deletePtFlag,bit29,events,timePt,time_lsw,type,xloc,yloc,deletePtFlag
4474//      FindValue/I=10 deletePtFlag             //bad points are flagged as 1, first real point is 10 (unless it was replaced)
4475//      Print V_Value                                                   //so this will fail...
4476       
4477        FindLevel/P/Q deletePtFlag 10
4478//      Print V_LevelX
4479       
4480        DeletePoints 0,trunc(V_LevelX)+1, bit29,events,timePt,time_lsw,type,xloc,yloc,deletePtFlag
4481
4482        SetDataFolder root:Packages:NIST:Event:
4483
4484// this is done after the file load/decode whether the XOP was used or the Igor code
4485// (don't do the time rescaling here)   
4486//      Duplicate/O timePt      rescaledTime
4487//      // MultiThread shaves off significant time!
4488//      MultiThread rescaledTime *= 1e-7                        //convert to seconds and that's all
4489
4490
4491// cleanup as many waves as possible to save space
4492// these can be kept as needed for debugging
4493        KillWaves/Z bit29,type,time_lsw,events,deletePtFlag
4494        KillWaves/Z badTimePt,badEventNum,PPTime,PPEventNum,T0Time,T0EventNum
4495        toc()
4496       
4497        SetDataFolder root:
4498        return(0)
4499end
4500
4501//Macro doDecodeOnly()
4502//      Variable numBad=0
4503//      DecodeEvents_New(type,events,bit29,xloc,yloc,time_lsw,numBad)
4504//End
4505
4506Function LoadEventAsHex(fname)
4507        String fname
4508       
4509        SetDataFolder root:Packages:NIST:Event:
4510       
4511        LoadWave/J/O/A/K=2/B="C=1,F=10,T=96,N=newWave;"  fname          //  /E=1 flag will display a table
4512       
4513        // table is useful to see as hexadecimal -- every other point
4514        // is FFFFFFFF = 4294967295 and is garbage to delete
4515       
4516        SetDataFolder root:
4517        return(0)
4518end
4519
4520// no need to remove every other point that is zero
4521// (actually 4294967295 = FFFFFFFF)
4522// I can to this with a wave assignment
4523Function RemoveFFF(w)
4524        Wave w
4525       
4526        SetDataFolder root:Packages:NIST:Event:
4527//      RemoveZeroEvents(w)
4528       
4529// 32 bit unsigned integer wave
4530        Make/O/U/I/N=(trunc(numpnts(w))/2) events
4531        MultiThread events = w[2*p]
4532       
4533        KillWaves/Z w           //not needed any longer
4534       
4535        SetDataFolder root:
4536        return(0)
4537End
4538
4539
4540
4541Function GetBitsFromEvents()
4542
4543//      Wave events=events
4544        SetDataFolder root:Packages:NIST:Event:
4545        Wave events = root:Packages:NIST:Event:events
4546               
4547        Make/O/U/B/N=(numpnts(events))  xloc,yloc,type,bit29
4548        Make/O/U/I/N=(numpnts(events)) time_lsw
4549
4550        MultiThread type = (events & 0xC0000000)/1073741824             //right shift by 2^30
4551       
4552        MultiThread bit29 = (events & 0x20000000)/536870912             //bit 29 only , shift by 2^29
4553
4554        MultiThread xloc = 127 - (events & 255)                                         //last 8 bits (7-0)
4555        MultiThread yloc = (events & 65280)/256                                         //bits 15-8, right shift by 2^8
4556
4557
4558        MultiThread time_lsw = (events & 536805376)/65536                       //13 bits, 28-16, right shift by 2^16
4559       
4560        Variable/G numXYevents,num0,num1,num2,num3
4561        num0 = CountType(0)
4562        num1 = CountType(1)
4563        num2 = CountType(2)
4564        num3 = CountType(3)
4565       
4566        numXYevents = num0 + num2
4567        Printf "numXYevents = type 0 + type 2 = %d\r",numXYevents
4568        Printf "XY = num0 = %d\r",num0
4569        Printf "time MSW = num1 = %d\r",num1
4570        Printf "XY time = num2 = %d\r",num2
4571        Printf "Rollover = num3 = %d\r",num3
4572
4573
4574// dispStr will be displayed on the panel
4575        SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
4576        SVAR filepathStr = root:Packages:NIST:Event:gEvent_logfile
4577       
4578        String tmpStr="",fileStr=""
4579        fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
4580       
4581        Variable fileref,totBytes
4582        Open/R fileref as filepathstr
4583                FStatus fileref
4584        Close fileref
4585
4586        totBytes = V_logEOF
4587       
4588        sprintf tmpStr, "%s: %d total bytes\r",fileStr,totBytes
4589        dispStr = tmpStr
4590        sprintf tmpStr,"numXYevents = %d\r",numXYevents
4591        dispStr += tmpStr
4592//      sprintf tmpStr,"PP = %d  :  ",numPP
4593//      dispStr += tmpStr
4594//      sprintf tmpStr,"ZeroData = %d\r",numZero
4595//      dispStr += tmpStr
4596        sprintf tmpStr,"Rollover = %d",num3
4597        dispStr += tmpStr
4598
4599       
4600//      Print 127 - (events[5] & 255)
4601//      Print (events[5] & 65280)/256
4602
4603        SetDataFolder root:
4604        return(0)
4605End
4606
4607//
4608// a quick way to count the number of a particular value in a wave
4609//
4610Function CountType(val)
4611        Variable val
4612       
4613        Wave type=type
4614        // can't duplicate, since Byte data can't accept NaN
4615        Make/O/D/N=(numpnts(type)) tmp
4616        MultiThread tmp = type
4617       
4618        MultiThread tmp = (tmp[p] == val) ? NaN : tmp[p]                                // replace matches with NaN
4619       
4620        WaveStats/Q tmp
4621       
4622        KillWaves/Z tmp
4623        return(V_numNaNs)
4624End
4625
4626Function CleanUpBeginning(type,events,bit29,xloc,yloc,time_lsw)
4627        Wave type,events,bit29,xloc,yloc,time_lsw
4628       
4629        // for all of the waves, remove points from the beginning up to the
4630        // first one with a type==2 so that I know that the
4631        // time has been properly reset to start
4632       
4633        variable ii,num
4634        num=numpnts(type)
4635        for(ii=0;ii<num;ii+=1)
4636                if(type[ii] == 2)
4637                        break
4638                endif
4639        endfor
4640       
4641        Print "Num bad removed from beginning = ",ii
4642        DeletePoints 0,ii, type,events,bit29,xloc,yloc,time_lsw
4643
4644        return(ii)
4645End
4646
4647
4648//
4649// for the bit shifts, see the decimal-binary conversion
4650// http://www.binaryconvert.com/convert_unsigned_int.html
4651//
4652//              K0 = 536870912
4653//              Print (K0 & 0x08000000)/134217728       //bit 27 only, shift by 2^27
4654//              Print (K0 & 0x10000000)/268435456               //bit 28 only, shift by 2^28
4655//              Print (K0 & 0x20000000)/536870912               //bit 29 only, shift by 2^29
4656//
4657// This is duplicated by the XOP, but the Igor code allows quick access to print out
4658// all of the gory details of the events and every little bit of them. the print
4659// statements and flags are kept for this reason, so the code is a bit messy.
4660//
4661//
4662//Static Constant ATXY = 0
4663//Static Constant ATXYM = 2
4664//Static Constant ATMIR = 1
4665//Static Constant ATMAR = 3
4666//
4667//
4668Function DecodeEvents_New(type,events,bit29,xloc,yloc,time_lsw,numBad)
4669        Wave type,events,bit29,xloc,yloc,time_lsw
4670        Variable numBad         // number of bad points previously removed from beginning of file
4671
4672//      NVAR time_msw = root:Packages:NIST:Event:gEvent_time_msw
4673//      NVAR time_lsw = root:Packages:NIST:Event:gEvent_time_lsw
4674        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
4675       
4676//      SVAR filepathstr = root:Packages:NIST:Event:gEvent_logfile
4677        SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
4678       
4679        SetDataFolder root:Packages:NIST:Event
4680
4681
4682        variable ii,num,typ
4683        Variable nRoll,roll_time,time_msw,timeval
4684        Variable tmpPP,tmpT0,numRemoved,rolloverHappened
4685        Variable tmpX,tmpY
4686
4687//      tic()
4688        // 32-bit unsigned, max value = 4,294,926,295 (= max number of events I can sort)
4689        Make/O/I/U/N=(numpnts(type))  deletePtFlag
4690        MultiThread deletePtFlag = p+10         // give them all a different number, starting from 10
4691        // flagged "bad" points will be set == 1
4692       
4693        Make/O/N=500000 badTimePt,badEventNum,PPTime,PPEventNum,T0Time,T0EventNum
4694        MultiThread badTimePt=0
4695        MultiThread badEventNum=0
4696        MultiThread PPTime=0
4697        MultiThread PPEventNum=0
4698        MultiThread T0Time=0
4699        MultiThread T0EventNum=0
4700
4701        tmpPP=0
4702        tmpT0=0
4703        numRemoved=0
4704       
4705        Make/O/D/N=(numpnts(time_lsw)) timePt
4706        MultiThread timePt = 0          //need DP wave for time, time_lsw is 32 bit int
4707
4708
4709        nRoll = 0               //number of rollover events
4710        roll_time = 2^26                //units of 10-7 sec
4711        time_msw=0
4712       
4713        NVAR removeBadEvents = root:Packages:NIST:Event:gRemoveBadEvents
4714       
4715        num=numpnts(type)
4716
4717//
4718//              NOTE: I now have the types in 0123 order in the switch - so they will
4719//              appear different than the old Igor code which was 0132
4720//
4721       
4722        for(ii=0;ii<num;ii+=1)
4723//              if(mod(ii,10000)==0)
4724//                      Print "step %g of %g",ii,num
4725//              endif
4726               
4727                typ = type[ii]
4728               
4729                switch(typ)
4730                        case ATXY:
4731                       
4732                                // if the datavalue is == 0, just skip it now (it can only be interpreted as type 0, obviously)
4733                                if(events[ii] == 0 && RemoveBadEvents == 1)
4734                                        numRemoved += 1
4735//                                      Print "zero at ii= ",ii
4736
4737// flag for deletion later
4738                                        deletePtFlag[ii] = 1
4739                                       
4740//                                      DeletePoints ii,1, type,events,bit29,xloc,yloc,time_lsw,timePt
4741//                                      num -= 1
4742//                                      ii -= 1
4743
4744                                        break           //don't increment ii
4745                                endif
4746                               
4747                                //if bit29=1, it's pileup, delete the point, decrement ii and num and break out
4748                                if(bit29[ii] == 1)
4749                                        PPTime[tmpPP] = timeval
4750                                        PPEventNum[tmpPP] = ii
4751                                        tmpPP += 1
4752                                        numRemoved += 1
4753                                        // flag for deletion later
4754                                        deletePtFlag[ii] = 1
4755//                                      DeletePoints ii,1, type,events,bit29,xloc,yloc,time_lsw,timePt
4756//                                      num -= 1
4757//                                      ii -= 1
4758                                        break
4759                                endif
4760                               
4761                                //otherwise the point is good, calculate the time
4762                                timePt[ii] = trunc( nRoll*roll_time + (time_msw * (8192)) + time_lsw[ii] )              //left shift msw by 2^13, then add in lsw, as an integer
4763                                if (timePt[ii] > t_longest)
4764                                        t_longest = timePt[ii]
4765                                endif
4766                               
4767                                // catch the "bad" events:
4768                                // if an XY event follows a rollover, time_msw is 0 by definition, but does not immediately get
4769                                // re-evalulated here. Throw out only the immediately following points where msw is still 8191
4770                                if(rolloverHappened && RemoveBadEvents == 1)
4771                                        // maybe a bad event, throw it out
4772                                        if(time_msw == 8191)
4773                                                badTimePt[numBad] = timeVal
4774                                                badEventNum[numBad] = ii
4775                                                numBad +=1
4776                                                numRemoved += 1
4777                                                // flag for deletion later
4778                                                deletePtFlag[ii] = 1
4779//                                              DeletePoints ii,1, type,events,bit29,xloc,yloc,time_lsw,timePt
4780//                                              num -= 1
4781//                                              ii -= 1
4782                                        else
4783                                                // time_msw has been reset, points are good now, so keep this one
4784                                                rolloverHappened = 0
4785                                        endif
4786                                endif
4787                               
4788                                break
4789
4790                        case ATMIR:             // 1
4791
4792                                time_msw =  (events[ii] & 536805376)/65536                      //13 bits, 28-16, right shift by 2^16
4793                                timePt[ii] = trunc( nRoll*roll_time + (time_msw * (8192)) + time_lsw[ii] )
4794                                if (timePt[ii] > t_longest)
4795                                        t_longest = timePt[ii]
4796                                endif
4797                               
4798                                if(bit29[ii] != 0)              // bit 29 set is a T0 event, not a rollover
4799                                        //Printf "bit29 = 1 at ii = %d : type = %d\r",ii,type
4800                                        T0Time[tmpT0] = time_lsw[ii]
4801                                        T0EventNum[tmpT0] = ii
4802                                        tmpT0 += 1
4803                                        // reset nRoll = 0 for calcluating the time
4804                                        nRoll = 0
4805                                endif
4806                               
4807                                // previous event was "2" (kept XY from there)
4808                                //
4809                                xloc[ii] = tmpX
4810                                yloc[ii] = tmpY
4811                               
4812                                break
4813                        case ATXYM:             // 2
4814       
4815                                //
4816                                // keep only the XY position, next event will be ATMIR
4817                                // where these XY points will be written
4818                                //
4819                                tmpX = xloc[ii]
4820                                tmpY = yloc[ii]
4821                                // flag for deletion later
4822                                deletePtFlag[ii] = 1
4823//                              DeletePoints ii,1, type,events,bit29,xloc,yloc,time_lsw ,timePt
4824//                              num -= 1
4825//                              ii -= 1
4826                               
4827                                // next event must be ATMIR (type 1) that will contain the MSW time bits
4828                                break
4829                        case ATMAR:             // 3
4830                       
4831                                nRoll += 1
4832                               
4833                                if(bit29[ii] != 0)              // bit 29 set is a T0 event, not a rollover
4834                                        //Printf "bit29 = 1 at ii = %d : type = %d\r",ii,type
4835                                        T0Time[tmpT0] = time_lsw[ii]
4836                                        T0EventNum[tmpT0] = ii
4837                                        tmpT0 += 1
4838                                        // reset nRoll = 0 for calcluating the time
4839                                        nRoll = 0
4840                                endif
4841                               
4842                                rolloverHappened = 1
4843                               
4844                                // delete the point since it's not XY
4845                                // flag for deletion later
4846                                deletePtFlag[ii] = 1
4847//                              DeletePoints ii,1, type,events,bit29,xloc,yloc,time_lsw ,timePt
4848//                              num -= 1
4849//                              ii -= 1
4850                                       
4851                                break
4852                        default:
4853                endswitch
4854       
4855        endfor
4856
4857//      printf("Igor new method full file decode done in  ")   
4858//      toc()
4859       
4860        Print "Events removed (Igor) = ",numRemoved
4861       
4862        Variable numXYEvents = numpnts(xloc)
4863        String tmpStr=""
4864       
4865        sPrintf tmpStr,"\rBad Rollover Events = %d (%4.4g %% of events)",numBad,numBad/numXYevents*100
4866        dispStr += tmpStr
4867        sPrintf tmpStr,"\rTotal Events Removed = %d (%4.4g %% of events)",numRemoved,numRemoved/numXYevents*100
4868        dispStr += tmpStr
4869        SetDataFolder root:
4870
4871
4872        return(0)
4873
4874end
4875
4876// not used any longer since the one-by-one deletion of points is
4877// WAY too slow to use for any event files of > 50 MB
4878Function RemoveZeroEvents(events)
4879        Wave events
4880       
4881        // start at the back and remove zeros
4882        Variable num=numpnts(events),ii,numToRemove,count
4883
4884        count = 0
4885        numToRemove = 2         //remove the zero, and the following FFFFFF
4886        ii=num
4887        do
4888                ii -= 1
4889                if(events[ii] == 0)
4890                        DeletePoints ii, numToRemove, events
4891                        count += 1
4892                endif
4893        while(ii > 0)
4894       
4895        print "removed zero events = ",count
4896        return(count)
4897End
4898
4899//////////////////////////////
Note: See TracBrowser for help on using the repository browser.