source: sans/Dev/trunk/NCNR_User_Procedures/Reduction/SANS/EventModeProcessing.ipf @ 1115

Last change on this file since 1115 was 1115, checked in by srkline, 4 years ago

attempting to put the zip file of the package in some accessible location since webster has been cut off.

File size: 118.0 KB
Line 
1#pragma rtGlobals=1             // Use modern global access method.
2#pragma IgorVersion=6.22
3
4// vers 7.13e
5
6// TODO:
7//
8// -- search for TODO for unresolved issues not on this list
9//
10// -- add comments to the code as needed
11//
12// -- write the help file, and link the help buttons to the help docs
13//
14// -- examples?
15//
16// -- added a test function AutoFix_Rollover_Steps() that will search for steps (simple delta) that
17//    are +/- 0.1s away from the 6.7s missed rollover. This function looks for up, down, then re-calculates
18//    the derivative. ?? should this be set to work only between the cursors? Currently it does the whole set.
19//
20// -- added a test function PutCursorsAtBigStep(tol). that will look for a big "jump", larger than tol.
21//    find a way to work this into the panel, maybe add a button and a setVar to set the value
22//
23//
24// -- EC_FindStepButton_down (and up) now both do the SAME thing -- using the "PutCursorsAtBigStep" function
25//    as above. This may change in the future...
26//
27//
28// -- NEW 2016
29//
30// -- see the Procedure DecodeBinaryEvents(numToPrint) for a first implementation of a "viewer"
31//    for the binary events. Prints out the decoded 32-bit binary to the command window for inspection
32//    Could be modified as needed to flag events, see what bad events really look like, etc. (May 2016)
33
34//  -- see the proc DisplayForSlicing() that may aid in setting the bins properly by plotting the
35//     bins along with the differential collection rate
36//
37// -- ?? need way to get correspondence between .hst files and VAX files? Names are all different. See
38//    DateAndTime2HSTName() functions and similar @ bottom of this file
39//
40// x- add the XOP to the distribution package
41//
42// x- Need to make sure that the rescaledTime and the differentiated time graphs are
43//     being properly updated when the data is processed, modified, etc.
44//
45// -- I need better nomenclature other than "stream" for the "continuous" data set.
46//     It's all a stream, just sometimes it's not oscillatory
47//
48//
49// X- the slice display "fails" for data sets that have 3 or 4 slices, as the ModifyImage command
50//     interprets the data as being RGB - and so does nothing.
51//     need to find a way around this. This was fixed by displaying the data using the G=1 flag on AppendImage
52//     to prevent the "atuo-detection" of data as RGB
53//
54// -- Do something with the PP events. Currently, only the PP events that are XY (just the
55//    type 0 events (since I still need to find out what they realy mean)
56//
57// -- Add a switch to allow Sorting of the Stream data to remove the "time-reversed" data
58//     points. Maybe not kosher, but would clean things up.
59//
60// -- when a large stream file is split, and each segment is edited, it is re-loaded from a list and is currently
61//    NOT decimated, and there is no option. There could be an option for this, especially for these
62//    large files. The issue is that the data segments are concatentated (adjusting the time) and can
63//    be too large to handle (a 420 MB event file split and edited and re-loaded becomes a 1.6 GB Igor experiment!!)
64//
65//    -- so allow the option of either decimation of the segments as they are loaded, or create the option to
66//       bin on the fly so that the full concatentated set does not need to be stored. Users could be
67//       directed to determine the binning and manipulate the data of a sufficiently decimated data set
68//       first to set the binning, then only process the full set.
69//
70///////////////////////////////
71//
72// NOTE -- to be able to show the T0 and PP event locations/times, force the loader to use the Igor code rather than
73// the XOP. Then there will be waves generated with their locations: T0Time and T0EventNum, PPTime and PPEventNum. Edit these waves
74// and (1) delete the zero points at the end of the waves and (2) multiply the Time wave * 1e-7 to convert to seconds.
75// Then the waves can be plotted on top of the event data, so it can be seen where these events were identified.
76//
77///////////////   SWITCHES     /////////////////
78//
79// for the "File Too Big" limit:
80//      Variable/G root:Packages:NIST:Event:gEventFileTooLarge = 150            // 150 MB considered too large
81//
82// for the tolerance of "step" detection
83//      Variable/G root:Packages:NIST:Event:gStepTolerance = 5          // 5 = # of standard deviations from mean. See PutCursorsAtStep()
84//
85//
86///////// DONE //////////
87//
88// X- memory issues:
89//              -- in LoadEvents -- should I change the MAKE to:
90//                              /I/U is unsigned 32-bit integer (for the time)
91//                              /B/U is unsigned 8-bit integer (max val=255) for the x and y values
92//                      -- then how does this affect downstream processing - such as rescaledTime, differentiation, etc.
93//                      x- and can I re-write the XOP to create these types of data waves, and properly fill them...
94//
95//  **- any integer waves must be translated by Igor into FP to be able to be displayed or for any
96//    type of analysis. so it's largely a waste of time to use integers. so simply force the XOP to
97//    generate only SP waves. this will at least save some space.
98//
99//
100//
101// x- Is there any way to improve the speed of the loader? How could an XOP be structured
102//     for maximum flexibility? Leave the post processing to Igor, but how much for the XOP
103//     to do? And can it handle such large amounts of data to pass back and forth, or
104//     does it need to be written as an operation, rather than a function??? I'd really
105//     rather that Igor handles the memory management, not me, if I write the XOP.
106//
107// **- as of 11/27, the OSX version of the XOP event loader is about 35x faster for the load!
108//    and is taking approx 1.8s/28MB, or about 6.5s/100MB of file. quite reasonable now, and
109//    probably a bit faster yet on the PC.
110//
111//
112// X- fix the log/lin display - it's not working correctly
113//                      I could use ModifyImage and log = 0|1 keyword for the log Z display
114//                      rather than creating a duplicate wave of log(data)
115//                      -- it's in the Function sliceSelectEvent_Proc()
116//
117// X- add controls to show the bar graph
118// x- add popup for selecting the binning type
119// x- add ability to save the slices to RAW VAX files
120// X- add control to show the bin counts and bin end times
121// x- ADD buttons, switches, etc for the oscillatory mode - so that this can be accessed
122//
123// x- How are the headers filled for the VAX files from Teabag???
124// -- I currently read the events 2x. Once to count the events to make the waves the proper
125//     size, then a second time to actualy process the events. Would it be faster to insert points
126//     as needed, or to estimate the size, and make it too large, then trim at the end...
127// ((( NO -- I have no good way of getting a proper estimate of how many XY events there are for a file))
128//
129//
130//
131
132
133//
134// These are also defined in the TISANE procedure file. In both files they are declared
135// as Static, so they are local to each procedure
136//
137Static Constant ATXY = 0
138Static Constant ATXYM = 2
139Static Constant ATMIR = 1
140Static Constant ATMAR = 3
141
142Static Constant USECSPERTICK=0.1 // microseconds
143Static Constant TICKSPERUSEC=10
144Static Constant XBINS=128
145Static Constant YBINS=128
146//
147
148Static Constant MODE_STREAM = 0
149Static Constant MODE_OSCILL = 1
150Static Constant MODE_TISANE = 2
151Static Constant MODE_TOF = 3
152
153//Menu "Macros"
154//      "Split Large File",SplitBigFile()
155//      "Accumulate First Slice",AccumulateSlices(0)
156//      "Add Current Slice",AccumulateSlices(1)
157//      "Display Accumulated Slices",AccumulateSlices(2)       
158//End
159
160
161
162Proc Show_Event_Panel()
163        DoWindow/F EventModePanel
164        if(V_flag ==0)
165                Init_Event()
166                EventModePanel()
167        EndIf
168End
169
170
171Function Init_Event()
172
173        NewDataFolder/O/S root:Packages:NIST:Event
174
175        String/G        root:Packages:NIST:Event:gEvent_logfile
176        String/G        root:Packages:NIST:Event:gEventDisplayString="Details of the file load"
177       
178        Variable/G      root:Packages:NIST:Event:AIMTYPE_XY=0 // XY Event
179        Variable/G      root:Packages:NIST:Event:AIMTYPE_XYM=2 // XY Minor event
180        Variable/G      root:Packages:NIST:Event:AIMTYPE_MIR=1 // Minor rollover event
181        Variable/G      root:Packages:NIST:Event:AIMTYPE_MAR=3 // Major rollover event
182
183        Variable/G root:Packages:NIST:Event:gEvent_time_msw = 0
184        Variable/G root:Packages:NIST:Event:gEvent_time_lsw = 0
185        Variable/G root:Packages:NIST:Event:gEvent_t_longest = 0
186
187        Variable/G root:Packages:NIST:Event:gEvent_tsdisp //Displayed slice
188        Variable/G root:Packages:NIST:Event:gEvent_nslices = 10  //Number of time slices
189       
190        Variable/G root:Packages:NIST:Event:gEvent_logint = 1
191
192        Variable/G root:Packages:NIST:Event:gEvent_Mode = MODE_OSCILL                           // ==0 for "stream", ==1 for Oscillatory
193        Variable/G root:Packages:NIST:Event:gRemoveBadEvents = 1                // ==1 to remove "bad" events, ==0 to read "as-is"
194        Variable/G root:Packages:NIST:Event:gSortStreamEvents = 0               // ==1 to sort the event stream, a last resort for a stream of data
195       
196        Variable/G root:Packages:NIST:Event:gEvent_ForceTmaxBin=1               //==1 to enforce t_longest in user-defined custom bins
197
198        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
199       
200               
201        Make/D/O/N=(XBINS,YBINS,nslices) slicedData
202        Duplicate/O slicedData logslicedData
203        Duplicate/O slicedData dispsliceData
204
205
206// for decimation
207        Variable/G root:Packages:NIST:Event:gEventFileTooLarge = 2500           // 2500 MB considered too large
208        Variable/G root:Packages:NIST:Event:gDecimation = 100
209        Variable/G root:Packages:NIST:Event:gEvent_t_longest_decimated = 0
210        Variable/G root:Packages:NIST:Event:gEvent_t_segment_start = 0
211        Variable/G root:Packages:NIST:Event:gEvent_t_segment_end = 0
212
213// for large file splitting
214        String/G root:Packages:NIST:Event:gSplitFileList = ""           // a list of the file names as split
215       
216// for editing
217        Variable/G root:Packages:NIST:Event:gStepTolerance = 5          // 5 = # of standard deviations from mean. See PutCursorsAtStep()
218       
219        SetDataFolder root:
220End
221
222//
223// -- extra bits of buttons... not used
224//
225//      Button button9 title="Decimation",size={100,20},pos={490,400},proc=E_ShowDecimateButton
226//
227//      Button button11,pos={490,245},size={150,20},proc=LoadDecimateButtonProc,title="Load and Decimate"
228//      Button button12,pos={490,277},size={150,20},proc=ConcatenateButtonProc,title="Concatenate"
229//      Button button13,pos={490,305},size={150,20},proc=DisplayConcatenatedButtonProc,title="Display Concatenated"
230//     
231//      GroupBox group0 title="Manual Controls",size={185,112},pos={490,220}
232//
233//      NewPanel /W=(82,44,854,664)/N=EventModePanel/K=2
234//      DoWindow/C EventModePanel
235//      ModifyPanel fixedSize=1,noEdit =1
236Proc EventModePanel()
237        PauseUpdate; Silent 1           // building window...
238        NewPanel /W=(82,44,884,664)/N=EventModePanel/K=2
239        DoWindow/C EventModePanel
240//      ModifyPanel fixedSize=1,noEdit =1
241
242        SetDrawLayer UserBack
243        DrawText 479,345,"Stream Data"
244        DrawLine 563,338,775,338
245        DrawText 479,419,"Oscillatory Data"
246        DrawLine 580,411,775,411
247
248//      ShowTools/A
249        Button button0,pos={14,87},size={150,20},proc=LoadEventLog_Button,title="Load Event Log File"
250        Button button0,fSize=12
251        TitleBox tb1,pos={475,500},size={266,86},fSize=10
252        TitleBox tb1,variable= root:Packages:NIST:Event:gEventDisplayString
253
254        CheckBox chkbox2,pos={376,151},size={81,15},proc=LogIntEvent_Proc,title="Log Intensity"
255        CheckBox chkbox2,fSize=10,variable= root:Packages:NIST:Event:gEvent_logint
256        CheckBox chkbox3,pos={14,125},size={119,15},title="Remove Bad Events?",fSize=10
257        CheckBox chkbox3,variable= root:Packages:NIST:Event:gRemoveBadEvents
258       
259        Button doneButton,pos={738,36},size={50,20},proc=EventDone_Proc,title="Done"
260        Button doneButton,fSize=12
261        Button button2,pos={486,200},size={140,20},proc=ShowEventDataButtonProc,title="Show Event Data"
262        Button button3,pos={486,228},size={140,20},proc=ShowBinDetailsButtonProc,title="Show Bin Details"
263        Button button5,pos={633,228},size={140,20},proc=ExportSlicesButtonProc,title="Export Slices as VAX"
264        Button button6,pos={748,9},size={40,20},proc=EventModeHelpButtonProc,title="?"
265               
266        Button button7,pos={211,33},size={120,20},proc=AdjustEventDataButtonProc,title="Adjust Events"
267        Button button8,pos={653,201},size={120,20},proc=CustomBinButtonProc,title="Custom Bins"
268        Button button4,pos={211,63},size={120,20},proc=UndoTimeSortButtonProc,title="Undo Time Sort"
269        Button button18,pos={211,90},size={120,20},proc=EC_ImportWavesButtonProc,title="Import Edited"
270       
271        SetVariable setvar0,pos={208,149},size={160,16},proc=sliceSelectEvent_Proc,title="Display Time Slice"
272        SetVariable setvar0,fSize=10
273        SetVariable setvar0,limits={0,1000,1},value= root:Packages:NIST:Event:gEvent_tsdisp     
274        SetVariable setvar1,pos={389,29},size={160,16},title="Number of slices",fSize=10
275        SetVariable setvar1,limits={1,1000,1},value= root:Packages:NIST:Event:gEvent_nslices
276        SetVariable setvar2,pos={389,54},size={160,16},title="Max Time (s)",fSize=10
277        SetVariable setvar2,value= root:Packages:NIST:Event:gEvent_t_longest
278       
279        PopupMenu popup0,pos={389,77},size={119,20},proc=BinTypePopMenuProc,title="Bin Spacing"
280        PopupMenu popup0,fSize=10
281        PopupMenu popup0,mode=1,popvalue="Equal",value= #"\"Equal;Fibonacci;Custom;\""
282        Button button1,pos={389,103},size={120,20},fSize=12,proc=ProcessEventLog_Button,title="Bin Event Data"
283
284        Button button10,pos={488,305},size={100,20},proc=SplitFileButtonProc,title="Split Big File"
285        Button button14,pos={488,350},size={120,20},proc=Stream_LoadDecim,title="Load+Decimate"
286        Button button19,pos={639,350},size={130,20},proc=Stream_LoadAdjustedList,title="Load+Concatenate"
287        Button button20,pos={680,305},size={90,20},proc=ShowList_ToLoad,title="Show List"
288
289        Button button21,pos={649,378},size={120,20},proc=Stream_LoadAdjList_BinOnFly,title="Load+Accumulate"
290
291        SetVariable setvar3,pos={487,378},size={150,16},title="Decimation factor"
292        SetVariable setvar3,fSize=10
293        SetVariable setvar3,limits={1,inf,1},value= root:Packages:NIST:Event:gDecimation
294
295        Button button15_0,pos={488,425},size={110,20},proc=AccumulateSlicesButton,title="Add First Slice"
296        Button button16_1,pos={488,450},size={110,20},proc=AccumulateSlicesButton,title="Add Next Slice"
297        Button button17_2,pos={620,425},size={110,20},proc=AccumulateSlicesButton,title="Display Total"
298        Button button22,pos={620,450},size={120,20},proc=Osc_LoadAdjList_BinOnFly,title="Load+Accumulate"
299
300        CheckBox chkbox1_0,pos={25,34},size={69,14},title="Oscillatory",fSize=10
301        CheckBox chkbox1_0,mode=1,proc=EventModeRadioProc,value=1
302        CheckBox chkbox1_1,pos={25,59},size={53,14},title="Stream",fSize=10
303        CheckBox chkbox1_1,proc=EventModeRadioProc,value=0,mode=1
304        CheckBox chkbox1_2,pos={104,59},size={53,14},title="TISANE",fSize=10
305        CheckBox chkbox1_2,proc=EventModeRadioProc,value=0,mode=1
306        CheckBox chkbox1_3,pos={104,34},size={37,14},title="TOF",fSize=10
307        CheckBox chkbox1_3,proc=EventModeRadioProc,value=0,mode=1
308       
309        GroupBox group0_0,pos={5,5},size={174,112},title="(1) Loading Mode",fSize=12,fStyle=1
310        GroupBox group0_1,pos={372,5},size={192,127},title="(3) Bin Events",fSize=12,fStyle=1
311        GroupBox group0_2,pos={477,169},size={310,92},title="(4) View / Export",fSize=12,fStyle=1
312        GroupBox group0_3,pos={191,5},size={165,117},title="(2) Edit Events",fSize=12,fStyle=1
313        GroupBox group0_4,pos={474,278},size={312,200},title="Split / Accumulate Files",fSize=12
314        GroupBox group0_4,fStyle=1
315       
316        Display/W=(10,170,460,610)/HOST=#
317        AppendImage/T/G=1 :Packages:NIST:Event:dispsliceData            //  /G=1 flag prevents interpretation as RGB so 3, 4 slices display correctly
318        ModifyImage dispsliceData ctab= {*,*,ColdWarm,0}
319        ModifyImage dispsliceData ctabAutoscale=3
320        ModifyGraph margin(left)=14,margin(bottom)=14,margin(top)=14,margin(right)=14
321        ModifyGraph mirror=2
322        ModifyGraph nticks=4
323        ModifyGraph minor=1
324        ModifyGraph fSize=9
325        ModifyGraph standoff=0
326        ModifyGraph tkLblRot(left)=90
327        ModifyGraph btLen=3
328        ModifyGraph tlOffset=-2
329        RenameWindow #,Event_slicegraph
330        SetActiveSubwindow ##
331EndMacro
332
333
334
335
336// mode selector
337//Static Constant MODE_STREAM = 0
338//Static Constant MODE_OSCILL = 1
339//Static Constant MODE_TISANE = 2
340//Static Constant MODE_TOF = 3
341//
342Function EventModeRadioProc(name,value)
343        String name
344        Variable value
345       
346        NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
347       
348        strswitch (name)
349                case "chkbox1_0":
350                        gEventModeRadioVal= MODE_OSCILL
351                        break
352                case "chkbox1_1":
353                        gEventModeRadioVal= MODE_STREAM
354                        break
355                case "chkbox1_2":
356                        gEventModeRadioVal= MODE_TISANE
357                        break
358                case "chkbox1_3":
359                        gEventModeRadioVal= MODE_TOF
360                        break
361        endswitch
362        CheckBox chkbox1_0,value= gEventModeRadioVal==MODE_OSCILL
363        CheckBox chkbox1_1,value= gEventModeRadioVal==MODE_STREAM
364        CheckBox chkbox1_2,value= gEventModeRadioVal==MODE_TISANE
365        CheckBox chkbox1_3,value= gEventModeRadioVal==MODE_TOF
366
367        return(0)
368End
369
370Function AdjustEventDataButtonProc(ba) : ButtonControl
371        STRUCT WMButtonAction &ba
372
373        switch( ba.eventCode )
374                case 2: // mouse up
375                        // click code here
376                        Execute "ShowEventCorrectionPanel()"
377                        //
378                        break
379                case -1: // control being killed
380                        break
381        endswitch
382
383        return 0
384End
385
386Function CustomBinButtonProc(ba) : ButtonControl
387        STRUCT WMButtonAction &ba
388
389        switch( ba.eventCode )
390                case 2: // mouse up
391                        // click code here
392                        Execute "Show_CustomBinPanel()"
393                        //
394                        break
395                case -1: // control being killed
396                        break
397        endswitch
398
399        return 0
400End
401
402
403Function ShowEventDataButtonProc(ba) : ButtonControl
404        STRUCT WMButtonAction &ba
405
406        switch( ba.eventCode )
407                case 2: // mouse up
408                        // click code here
409                        Execute "ShowRescaledTimeGraph()"
410                        //
411                        DifferentiatedTime()
412                        //
413                        break
414                case -1: // control being killed
415                        break
416        endswitch
417
418        return 0
419End
420
421Function BinTypePopMenuProc(pa) : PopupMenuControl
422        STRUCT WMPopupAction &pa
423
424        switch( pa.eventCode )
425                case 2: // mouse up
426                        Variable popNum = pa.popNum
427                        String popStr = pa.popStr
428                        if(cmpstr(popStr,"Custom")==0)
429                                Execute "Show_CustomBinPanel()"
430                        endif
431                        break
432                case -1: // control being killed
433                        break
434        endswitch
435
436        return 0
437End
438
439Function ShowBinDetailsButtonProc(ba) : ButtonControl
440        STRUCT WMButtonAction &ba
441
442        switch( ba.eventCode )
443                case 2: // mouse up
444                        // click code here
445                        Execute "ShowBinTable()"
446                        Execute "BinEventBarGraph()"
447                        break
448                case -1: // control being killed
449                        break
450        endswitch
451
452        return 0
453End
454
455Function UndoTimeSortButtonProc(ba) : ButtonControl
456        STRUCT WMButtonAction &ba
457
458        switch( ba.eventCode )
459                case 2: // mouse up
460                        // click code here
461                        Execute "UndoTheSorting()"
462                        break
463                case -1: // control being killed
464                        break
465        endswitch
466
467        return 0
468End
469
470Function ExportSlicesButtonProc(ba) : ButtonControl
471        STRUCT WMButtonAction &ba
472
473        switch( ba.eventCode )
474                case 2: // mouse up
475                        // click code here
476                        Execute "ExportSlicesAsVAX()"           //will invoke the dialog
477                        break
478                case -1: // control being killed
479                        break
480        endswitch
481
482        return 0
483End
484
485Function EventModeHelpButtonProc(ba) : ButtonControl
486        STRUCT WMButtonAction &ba
487
488        switch( ba.eventCode )
489                case 2: // mouse up
490                        // click code here
491                        DisplayHelpTopic/Z "Event Mode Data"
492                        break
493                case -1: // control being killed
494                        break
495        endswitch
496
497        return 0
498End
499
500
501Function EventDone_Proc(ba) : ButtonControl
502        STRUCT WMButtonAction &ba
503       
504        String win = ba.win
505        switch (ba.eventCode)
506                case 2:
507                        DoWindow/K EventModePanel
508                        break
509        endswitch
510        return(0)
511End
512
513
514
515Function ProcessEventLog_Button(ctrlName) : ButtonControl
516        String ctrlName
517       
518        NVAR mode=root:Packages:NIST:Event:gEvent_Mode
519       
520        if(mode == MODE_STREAM)
521                Stream_ProcessEventLog("")
522        endif
523       
524        if(mode == MODE_OSCILL)
525                Osc_ProcessEventLog("")
526        endif
527       
528        // If TOF mode, process as Oscillatory -- that is, take the times as is
529        if(mode == MODE_TOF)
530                Osc_ProcessEventLog("")
531        endif
532       
533        // toggle the checkbox for log display to force the display to be correct
534        NVAR gLog = root:Packages:NIST:Event:gEvent_logint
535        LogIntEvent_Proc("",gLog)
536       
537        return(0)
538end
539
540// for oscillatory mode
541//
542Function Osc_ProcessEventLog(ctrlName)
543        String ctrlName
544
545        Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
546       
547        Wave binnedData = root:Packages:NIST:Event:binnedData
548        Wave xLoc = root:Packages:NIST:Event:xLoc
549        Wave yLoc = root:Packages:NIST:Event:yLoc
550
551// now with the number of slices and max time, process the events
552
553        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
554        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
555
556        SetDataFolder root:Packages:NIST:Event          //don't count on the folder remaining here
557       
558        Make/D/O/N=(XBINS,YBINS,nslices) slicedData
559               
560        Wave slicedData = slicedData
561        Wave rescaledTime = rescaledTime
562        Wave timePt = timePt
563        Make/O/D/N=(128,128) tmpData
564        Make/O/D/N=(nslices+1) binEndTime,binCount
565        Make/O/D/N=(nslices) timeWidth
566        Wave timeWidth = timeWidth
567        Wave binEndTime = binEndTime
568        Wave binCount = binCount
569
570        variable ii,del,p1,p2,t1,t2
571        del = t_longest/nslices
572
573        slicedData = 0
574        binEndTime[0]=0
575        BinCount[nslices]=0
576
577
578        String binTypeStr=""
579        ControlInfo /W=EventModePanel popup0
580        binTypeStr = S_value
581       
582        strswitch(binTypeStr)   // string switch
583                case "Equal":           // execute if case matches expression
584                        SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
585                        break                                           // exit from switch
586                case "Fibonacci":               // execute if case matches expression
587                        SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
588                        break
589                case "Log":             // execute if case matches expression
590                        SetLogBins(binEndTime,timeWidth,nslices,t_longest)
591                        break
592                case "Custom":          // execute if case matches expression
593                        //bins are set by the user on the panel - assume it's good to go
594                        break
595                default:                                                        // optional default expression executed
596                        DoAlert 0,"No match for bin type, Equal bins used"
597                        SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
598        endswitch
599
600
601// now before binning, sort the data
602
603        //MakeIndex is REALLY SLOW - this is slow - to allow undoing the sorting and starting over, but if you don't,
604        // you'll never be able to undo the sort
605        //
606        //
607        // for a 500 MB hst, MakeIndex = 83s, everything else in this function totalled < 5s
608        //
609        SetDataFolder root:Packages:NIST:Event:
610
611        if(WaveExists($"root:Packages:NIST:Event:OscSortIndex") == 0 )
612                Duplicate/O rescaledTime OscSortIndex
613//              tic()
614                MakeIndex rescaledTime OscSortIndex
615//              toc()
616//              tic()
617                IndexSort OscSortIndex, yLoc,xLoc,timePt,rescaledTime   
618//              toc()
619                //SetDataFolder root:Packages:NIST:Event
620//              tic()
621                IndexForHistogram(xLoc,yLoc,binnedData)                 // index the events AFTER sorting
622//              toc()
623                //SetDataFolder root:
624        Endif
625       
626        Wave index = root:Packages:NIST:Event:SavedIndex                //this is the histogram index
627
628//      tic()
629        for(ii=0;ii<nslices;ii+=1)
630                if(ii==0)
631//                      t1 = ii*del
632//                      t2 = (ii+1)*del
633                        p1 = BinarySearch(rescaledTime,0)
634                        p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
635                else
636//                      t2 = (ii+1)*del
637                        p1 = p2+1               //one more than the old one
638                        p2 = BinarySearch(rescaledTime,binEndTime[ii+1])               
639                endif
640
641        // typically zero will never be a valid time value in oscillatory mode. in "stream" mode, the first is normalized to == 0
642        // but not here - times are what they are.
643                if(p1 == -1)
644                        Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
645                        p1 = 0          //set to the first point if it's off the end
646                Endif
647               
648                if(p2 == -2)
649                        Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
650                        p2 = numpnts(rescaledTime)-1            //set to the last point if it's off the end
651                Endif
652//              Print p1,p2
653
654
655                tmpData=0
656                JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
657                slicedData[][][ii] = tmpData[p][q]
658               
659//              binEndTime[ii+1] = t2
660                binCount[ii] = sum(tmpData,-inf,inf)
661        endfor
662
663//      toc()
664        Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
665        Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
666        logslicedData = log(slicedData)
667
668        SetDataFolder root:
669        return(0)
670End
671
672// for a "continuous exposure"
673//
674// if there is a sort of these events, I need to re-index the events for the histogram
675// - see the oscillatory mode  - and sort the events here, then immediately re-index for the histogram
676// - but with the added complication that I need to always remember to index for the histogram, every time
677// - since I don't know if I've sorted or un-sorted. Osc mode always forces a re-sort and a re-index
678//
679Function Stream_ProcessEventLog(ctrlName)
680        String ctrlName
681
682//      NVAR slicewidth = root:Packages:NIST:gTISANE_slicewidth
683
684       
685        Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
686       
687        Wave binnedData = root:Packages:NIST:Event:binnedData
688        Wave xLoc = root:Packages:NIST:Event:xLoc
689        Wave yLoc = root:Packages:NIST:Event:yLoc
690
691// now with the number of slices and max time, process the events
692
693        NVAR yesSortStream = root:Packages:NIST:Event:gSortStreamEvents         //do I sort the events?
694        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
695        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
696
697        SetDataFolder root:Packages:NIST:Event          //don't count on the folder remaining here
698       
699        Make/D/O/N=(XBINS,YBINS,nslices) slicedData
700               
701        Wave slicedData = slicedData
702        Wave rescaledTime = rescaledTime
703        Make/O/D/N=(128,128) tmpData
704        Make/O/D/N=(nslices+1) binEndTime,binCount//,binStartTime
705        Make/O/D/N=(nslices) timeWidth
706        Wave binEndTime = binEndTime
707        Wave timeWidth = timeWidth
708        Wave binCount = binCount
709
710        variable ii,del,p1,p2,t1,t2
711        del = t_longest/nslices
712
713        slicedData = 0
714        binEndTime[0]=0
715        BinCount[nslices]=0
716       
717        String binTypeStr=""
718        ControlInfo /W=EventModePanel popup0
719        binTypeStr = S_value
720       
721        strswitch(binTypeStr)   // string switch
722                case "Equal":           // execute if case matches expression
723                        SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
724                        break                                           // exit from switch
725                case "Fibonacci":               // execute if case matches expression
726                        SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
727                        break
728                case "Log":             // execute if case matches expression
729                        SetLogBins(binEndTime,timeWidth,nslices,t_longest)
730                        break
731                case "Custom":          // execute if case matches expression
732                        //bins are set by the user on the panel - assume it's good to go
733                        break
734                default:                                                        // optional default expression executed
735                        DoAlert 0,"No match for bin type, Equal bins used"
736                        SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
737        endswitch
738
739// TODO
740// the global exists for this switch, but it is not implemented - not sure whether
741// it's correct to implement this at all --
742//
743        if(yesSortStream == 1)
744                SortTimeData()
745        endif
746       
747// index the events before binning
748// if there is a sort of these events, I need to re-index the events for the histogram
749//      SetDataFolder root:Packages:NIST:Event
750        IndexForHistogram(xLoc,yLoc,binnedData)
751//      SetDataFolder root:
752        Wave index = root:Packages:NIST:Event:SavedIndex                //the index for the histogram
753       
754       
755        for(ii=0;ii<nslices;ii+=1)
756                if(ii==0)
757//                      t1 = ii*del
758//                      t2 = (ii+1)*del
759                        p1 = BinarySearch(rescaledTime,0)
760                        p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
761                else
762//                      t2 = (ii+1)*del
763                        p1 = p2+1               //one more than the old one
764                        p2 = BinarySearch(rescaledTime,binEndTime[ii+1])               
765                endif
766
767                if(p1 == -1)
768                        Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
769                        p1 = 0          //set to the first point if it's off the end
770                Endif
771                if(p2 == -2)
772                        Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
773                        p2 = numpnts(rescaledTime)-1            //set to the last point if it's off the end
774                Endif
775//              Print p1,p2
776
777
778                tmpData=0
779                JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
780                slicedData[][][ii] = tmpData[p][q]
781               
782//              binEndTime[ii+1] = t2
783                binCount[ii] = sum(tmpData,-inf,inf)
784        endfor
785
786        Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
787        Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
788        logslicedData = log(slicedData)
789
790        SetDataFolder root:
791        return(0)
792End
793
794
795Proc    UndoTheSorting()
796        Osc_UndoSort()
797End
798
799// for oscillatory mode
800//
801// -- this takes the previously generated index, and un-sorts the data to restore to the
802// "as-collected" state
803//
804Function Osc_UndoSort()
805
806        SetDataFolder root:Packages:NIST:Event          //don't count on the folder remaining here
807        Wave rescaledTime = rescaledTime
808        Wave OscSortIndex = OscSortIndex
809        Wave yLoc = yLoc
810        Wave xLoc = xLoc
811        Wave timePt = timePt
812
813        Sort OscSortIndex OscSortIndex,yLoc,xLoc,timePt,rescaledTime
814
815        KillWaves/Z OscSortIndex
816       
817        SetDataFolder root:
818        return(0)
819End
820
821
822// now before binning, sort the data
823//
824//this is slow - undoing the sorting and starting over, but if you don't,
825// you'll never be able to undo the sort
826//
827Function SortTimeData()
828
829
830        SetDataFolder root:Packages:NIST:Event:
831
832        KillWaves/Z OscSortIndex
833       
834        if(WaveExists($"root:Packages:NIST:Event:OscSortIndex") == 0 )
835                Duplicate/O rescaledTime OscSortIndex
836                MakeIndex rescaledTime OscSortIndex
837                IndexSort OscSortIndex, yLoc,xLoc,timePt,rescaledTime   
838        Endif
839       
840        SetDataFolder root:
841        return(0)
842End
843
844
845
846Function SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
847        Wave binEndTime,timeWidth
848        Variable nslices,t_longest
849
850        Variable del,ii,t2
851        binEndTime[0]=0         //so the bar graph plots right...
852        del = t_longest/nslices
853       
854        for(ii=0;ii<nslices;ii+=1)
855                t2 = (ii+1)*del
856                binEndTime[ii+1] = t2
857        endfor
858        binEndTime[ii+1] = t_longest*(1-1e-6)           //otherwise floating point errors such that the last time point is off the end of the Binary search
859
860        timeWidth = binEndTime[p+1]-binEndTime[p]
861
862        return(0)       
863End
864
865// TODO
866// either get this to work, or scrap it entirely. it currently isn't on the popup
867// so it can't be accessed
868Function SetLogBins(binEndTime,timeWidth,nslices,t_longest)
869        Wave binEndTime,timeWidth
870        Variable nslices,t_longest
871
872        Variable tMin,ii
873
874        Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
875       
876        binEndTime[0]=0         //so the bar graph plots right...
877
878        // just like the log-scaled q-points
879        tMin = rescaledTime[1]/1                        //just a guess... can't use tMin=0, and rescaledTime[0] == 0 by definition
880        Print rescaledTime[1], tMin
881        for(ii=0;ii<nslices;ii+=1)
882                binEndTime[ii+1] =alog(log(tMin) + (ii+1)*((log(t_longest)-log(tMin))/nslices))
883        endfor
884        binEndTime[ii+1] = t_longest            //otherwise floating point errors such that the last time point is off the end of the Binary search
885       
886        timeWidth = binEndTime[p+1]-binEndTime[p]
887
888        return(0)
889End
890
891Function MakeFibonacciWave(w,num)
892        Wave w
893        Variable num
894
895        //skip the initial zero
896        Variable f1,f2,ii
897        f1=1
898        f2=1
899        w[0] = f1
900        w[1] = f2
901        for(ii=2;ii<num;ii+=1)
902                w[ii] = f1+f2
903                f1=f2
904                f2=w[ii]
905        endfor
906               
907        return(0)
908end
909
910Function SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
911        Wave binEndTime,timeWidth
912        Variable nslices,t_longest
913
914        Variable tMin,ii,total,t2,tmp
915        Make/O/D/N=(nslices) fibo
916        fibo=0
917        MakeFibonacciWave(fibo,nslices)
918       
919//      Make/O/D tmpFib={1,1,2,3,5,8,13,21,34,55,89,144,233,377,610,987,1597,2584,4181,6765,10946}
920
921        binEndTime[0]=0         //so the bar graph plots right...
922        total = sum(fibo,0,nslices-1)           //total number of "pieces"
923       
924        tmp=0
925        for(ii=0;ii<nslices;ii+=1)
926                t2 = sum(fibo,0,ii)/total*t_longest
927                binEndTime[ii+1] = t2
928        endfor
929        binEndTime[ii+1] = t_longest            //otherwise floating point errors such that the last time point is off the end of the Binary search
930       
931        timeWidth = binEndTime[p+1]-binEndTime[p]
932       
933        return(0)
934End
935
936
937
938// TODO:
939//
940// ** currently, the "stream" loader uses the first data point as time=0
941//    and rescales everything to that time. "Osc" loading uses the times "as-is"
942//    from the file, trusting the times to be correct.
943//
944// Would TISANE or TOF need a different loader?
945//     
946Function LoadEventLog_Button(ctrlName) : ButtonControl
947        String ctrlName
948
949        NVAR mode=root:Packages:NIST:Event:gEvent_mode
950        Variable err=0
951        Variable fileref,totBytes
952        NVAR fileTooLarge = root:Packages:NIST:Event:gEventFileTooLarge         //limit load to 150MB
953
954        SVAR filename = root:Packages:NIST:Event:gEvent_logfile
955        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
956        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
957       
958        String fileFilters = "All Files:.*;Data Files (*.txt):.txt;"
959        String abortStr
960       
961        PathInfo/S catPathName
962        if(V_flag==0)
963                DoAlert 0,"Please 'Pick Path' to the data from the Main (yellow) Panel."
964                return(0)
965        endif
966       
967       
968        Open/R/D/P=catPathName/F=fileFilters fileref
969        filename = S_filename
970        if(strlen(S_filename) == 0)
971                // user cancelled
972                DoAlert 0,"No file selected, no file loaded."
973                return(1)
974        endif
975       
976/// Abort if the files are too large
977        Open/R fileref as fileName
978                FStatus fileref
979        Close fileref
980
981        totBytes = V_logEOF/1e6         //in MB
982        if(totBytes > fileTooLarge)
983                sprintf abortStr,"File is %g MB, larger than the limit of %g MB. Split and Decimate.",totBytes,fileTooLarge
984                Abort abortStr
985        endif
986       
987
988#if (exists("EventLoadWave")==4)
989        LoadEvents_XOP()
990#else
991        // XOP is not present, warn the user to re-run the installer
992        //check the 32-bit or 64-bit
993        String igorKindStr = StringByKey("IGORKIND", IgorInfo(0) )
994        String alertStr
995        if(strsearch(igorKindStr, "64", 0 ) != -1)
996                alertStr = "The Event Loader XOP is not installed for the 64-bit version of Igor. Without it, event loading will "
997                alertStr += "be slow. It is recommended that you re-run the NCNR Installer. Click YES to stop and "
998                alertStr += "do the installation, or NO to continue with the file loading."
999        else
1000                alertStr = "The Event Loader XOP is not installed for the 32-bit version of Igor. Without it, event loading will "
1001                alertStr += "be slow. It is recommended that you re-run the NCNR Installer. Click YES to stop and "
1002                alertStr += "do the installation, or NO to continue with the file loading."
1003        endif
1004        DoAlert 1,alertStr
1005       
1006        if(V_flag == 1)
1007                // get out gracefully
1008                SetDataFolder root:
1009                return(0)
1010        endif
1011       
1012        LoadEvents()
1013#endif 
1014
1015        SetDataFolder root:Packages:NIST:Event:
1016
1017//tic()
1018        Wave timePt=timePt
1019        Wave xLoc=xLoc
1020        Wave yLoc=yLoc
1021        CleanupTimes(xLoc,yLoc,timePt)          //remove zeroes
1022//toc()
1023       
1024       
1025/////
1026// now do a little processing of the times based on the type of data
1027//     
1028        if(mode == MODE_STREAM)         // continuous "Stream" mode - start from zero
1029                Duplicate/O timePt rescaledTime
1030                rescaledTime = 1e-7*(timePt-timePt[0])          //convert to seconds and start from zero
1031                t_longest = waveMax(rescaledTime)               //should be the last point     
1032        endif
1033       
1034        if(mode == MODE_OSCILL)         // oscillatory mode - don't adjust the times, we get periodic t0 to reset t=0
1035                Duplicate/O timePt rescaledTime
1036                rescaledTime *= 1e-7                    //convert to seconds and that's all
1037                t_longest = waveMax(rescaledTime)               //if oscillatory, won't be the last point, so get it this way
1038       
1039                KillWaves/Z OscSortIndex                        //to make sure that there is no old index hanging around
1040        endif
1041
1042// MODE_TISANE
1043//TODO -- tisane doesn't do anything different than oscillatory. Is this really correct??
1044        if(mode == MODE_TISANE)         // TISANE mode - don't adjust the times, we get periodic t0 to reset t=0
1045                Duplicate/O timePt rescaledTime
1046                rescaledTime *= 1e-7                    //convert to seconds and that's all
1047                t_longest = waveMax(rescaledTime)               //if oscillatory, won't be the last point, so get it this way
1048       
1049                KillWaves/Z OscSortIndex                        //to make sure that there is no old index hanging around
1050        endif
1051       
1052// MODE_TOF
1053        if(mode == MODE_TOF)            // TOF mode - don't adjust the times, we get periodic t0 to reset t=0
1054                Duplicate/O timePt rescaledTime
1055                rescaledTime *= 1e-7                    //convert to seconds and that's all
1056                t_longest = waveMax(rescaledTime)               //if oscillatory, won't be the last point, so get it this way
1057       
1058                KillWaves/Z OscSortIndex                        //to make sure that there is no old index hanging around
1059        endif
1060
1061        SetDataFolder root:
1062
1063        STRUCT WMButtonAction ba
1064        ba.eventCode = 2
1065        ShowEventDataButtonProc(ba)
1066
1067        SetDataFolder root:
1068
1069        return(0)
1070End
1071
1072//// for the mode of "one continuous exposure"
1073////
1074//Function Stream_LoadEventLog(ctrlName)
1075//      String ctrlName
1076//     
1077//      Variable fileref
1078//
1079//      SVAR filename = root:Packages:NIST:Event:gEvent_logfile
1080//      NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
1081//      NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1082//     
1083//      String fileFilters = "All Files:.*;Data Files (*.txt):.txt;"
1084//     
1085//      Open/R/D/F=fileFilters fileref
1086//      filename = S_filename
1087//      if(strlen(S_filename) == 0)
1088//              // user cancelled
1089//              DoAlert 0,"No file selected, no file loaded."
1090//              return(1)
1091//      endif
1092//
1093//#if (exists("EventLoadWave")==4)
1094//      LoadEvents_XOP()
1095//#else
1096//      LoadEvents()
1097//#endif       
1098//
1099//      SetDataFolder root:Packages:NIST:Event:
1100//
1101////tic()
1102//      Wave timePt=timePt
1103//      Wave xLoc=xLoc
1104//      Wave yLoc=yLoc
1105//      CleanupTimes(xLoc,yLoc,timePt)          //remove zeroes
1106//     
1107////toc()
1108//
1109//      Duplicate/O timePt rescaledTime
1110//      rescaledTime = 1e-7*(timePt-timePt[0])          //convert to seconds and start from zero
1111//      t_longest = waveMax(rescaledTime)               //should be the last point
1112//
1113//      SetDataFolder root:
1114//
1115//      return(0)
1116//End
1117//
1118//// for the mode "oscillatory"
1119////
1120//Function Osc_LoadEventLog(ctrlName)
1121//      String ctrlName
1122//     
1123//      Variable fileref
1124//
1125//      SVAR filename = root:Packages:NIST:Event:gEvent_logfile
1126//      NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
1127//      NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1128//     
1129//      String fileFilters = "All Files:.*;Data Files (*.txt):.txt;"
1130//     
1131//      Open/R/D/F=fileFilters fileref
1132//      filename = S_filename
1133//              if(strlen(S_filename) == 0)
1134//              // user cancelled
1135//              DoAlert 0,"No file selected, no file loaded."
1136//              return(1)
1137//      endif
1138//     
1139//#if (exists("EventLoadWave")==4)
1140//      LoadEvents_XOP()
1141//#else
1142//      LoadEvents()
1143//#endif       
1144//     
1145//      SetDataFolder root:Packages:NIST:Event:
1146//
1147//      Wave timePt=timePt
1148//      Wave xLoc=xLoc
1149//      Wave yLoc=yLoc
1150//      CleanupTimes(xLoc,yLoc,timePt)          //remove zeroes
1151//     
1152//      Duplicate/O timePt rescaledTime
1153//      rescaledTime *= 1e-7                    //convert to seconds and that's all
1154//      t_longest = waveMax(rescaledTime)               //if oscillatory, won't be the last point, so get it this way
1155//
1156//      KillWaves/Z OscSortIndex                        //to make sure that there is no old index hanging around
1157//
1158//      SetDataFolder root:
1159//
1160//      return(0)
1161//End
1162
1163
1164//
1165// -- MUCH faster to count the number of lines to remove, then delete (N)
1166// rather then delete them one-by-one in the do-loop
1167Function CleanupTimes(xLoc,yLoc,timePt)
1168        Wave xLoc,yLoc,timePt
1169
1170        // start at the back and remove zeros
1171        Variable num=numpnts(xLoc),ii,numToRemove
1172
1173        numToRemove = 0
1174        ii=num
1175        do
1176                ii -= 1
1177                if(timePt[ii] == 0 && xLoc[ii] == 0 && yLoc[ii] == 0)
1178                        numToRemove += 1
1179                endif
1180        while(timePt[ii-1] == 0 && xLoc[ii-1] == 0 && yLoc[ii-1] == 0)
1181       
1182        if(numToRemove != 0)
1183                DeletePoints ii, numToRemove, xLoc,yLoc,timePt
1184        endif
1185       
1186        return(0)
1187End
1188
1189Function LogIntEvent_Proc(ctrlName,checked) : CheckBoxControl
1190        String ctrlName
1191        Variable checked
1192               
1193        SetDataFolder root:Packages:NIST:Event
1194       
1195        Wave slicedData = slicedData
1196        Wave logSlicedData = logSlicedData
1197        Wave dispSliceData = dispSliceData
1198       
1199        if(checked)
1200                logslicedData = log(slicedData)
1201                Duplicate/O logslicedData dispsliceData
1202        else
1203                Duplicate/O slicedData dispsliceData
1204        endif
1205
1206        NVAR selectedslice = root:Packages:NIST:Event:gEvent_tsdisp
1207
1208        sliceSelectEvent_Proc("", selectedslice, "", "")
1209
1210        SetDataFolder root:
1211
1212End
1213
1214
1215// TODO (DONE)
1216// this "fails" for data sets that have 3 or 4 slices, as the ModifyImage command
1217// interprets the data as being RGB - and so does nothing.
1218// need to find a way around this
1219//
1220////  When first plotted, AppendImage/G=1 flag prevents interpretation as RGB so 3, 4 slices display correctly
1221///
1222// I could modify this procedure to use the log = 0|1 keyword for the log Z display
1223// rather than creating a duplicate wave of log(data)
1224//
1225Function sliceSelectEvent_Proc(ctrlName, varNum, varStr, varName) : SetVariableControl
1226        String ctrlName
1227        Variable varNum
1228        String varStr
1229        String varName
1230       
1231        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
1232        NVAR selectedslice = root:Packages:NIST:Event:gEvent_tsdisp
1233       
1234        if(varNum < 0)
1235                selectedslice = 0
1236                DoUpdate
1237        elseif (varNum > nslices-1)
1238                selectedslice = nslices-1
1239                DoUpdate
1240        else
1241                ModifyImage/W=EventModePanel#Event_slicegraph ''#0 plane = varNum
1242        endif
1243
1244End
1245
1246Function DifferentiatedTime()
1247
1248        Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
1249
1250        SetDataFolder root:Packages:NIST:Event:
1251               
1252        Differentiate rescaledTime/D=rescaledTime_DIF
1253//      Display rescaledTime,rescaledTime_DIF
1254        DoWindow/F Differentiated_Time
1255        if(V_flag == 0)
1256                Display/N=Differentiated_Time/K=1 rescaledTime_DIF
1257                Legend
1258                Modifygraph gaps=0
1259                ModifyGraph zero(left)=1
1260                Label left "\\Z14Delta (dt/event)"
1261                Label bottom "\\Z14Event number"
1262        endif
1263       
1264        SetDataFolder root:
1265       
1266        return(0)
1267End
1268
1269
1270//
1271// for the bit shifts, see the decimal-binary conversion
1272// http://www.binaryconvert.com/convert_unsigned_int.html
1273//
1274//              K0 = 536870912
1275//              Print (K0 & 0x08000000)/134217728       //bit 27 only, shift by 2^27
1276//              Print (K0 & 0x10000000)/268435456               //bit 28 only, shift by 2^28
1277//              Print (K0 & 0x20000000)/536870912               //bit 29 only, shift by 2^29
1278//
1279// This is duplicated by the XOP, but the Igor code allows quick access to print out
1280// all of the gory details of the events and every little bit of them. the print
1281// statements and flags are kept for this reason, so the code is a bit messy.
1282//
1283Function LoadEvents()
1284       
1285        NVAR time_msw = root:Packages:NIST:Event:gEvent_time_msw
1286        NVAR time_lsw = root:Packages:NIST:Event:gEvent_time_lsw
1287        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1288       
1289        SVAR filepathstr = root:Packages:NIST:Event:gEvent_logfile
1290        SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
1291       
1292       
1293////    Variable decFac = 10                    //decimation factor
1294////    Variable jj,keep
1295       
1296        SetDataFolder root:Packages:NIST:Event
1297
1298        Variable fileref
1299        String buffer
1300        String fileStr,tmpStr
1301        Variable dataval,timeval,type,numLines,verbose,verbose3
1302        Variable xval,yval,rollBit,nRoll,roll_time,bit29,bit28,bit27
1303        Variable ii,flaggedEvent,rolloverHappened,numBad=0,tmpPP=0,tmpT0=0
1304        Variable Xmax, yMax
1305       
1306        xMax = 127              // number the detector from 0->127
1307        yMax = 127
1308       
1309        verbose3 = 0                    //prints out the rollover events (type==3)
1310        verbose = 0
1311        numLines = 0
1312
1313       
1314        // what I really need is the number of XY events
1315        Variable numXYevents,num1,num2,num3,num0,totBytes,numPP,numT0,numDL,numFF,numZero
1316        Variable numRemoved
1317        numXYevents = 0
1318        num0 = 0
1319        num1 = 0
1320        num2 = 0
1321        num3 = 0
1322        numPP = 0
1323        numT0 = 0
1324        numDL = 0
1325        numFF = 0
1326        numZero = 0
1327        numRemoved = 0
1328
1329//tic()
1330        Open/R fileref as filepathstr
1331                FStatus fileref
1332        Close fileref
1333
1334        totBytes = V_logEOF
1335        Print "total bytes = ", totBytes
1336        if(EventDataType(filePathStr) != 1)             //if not hst extension of raw data
1337                Abort "Data must be raw event data with file extension .hst for this operation"
1338                return(0)
1339        endif   
1340//toc()
1341//
1342
1343
1344// do a "pre-scan to get some of the counts, so that I can allocate space. This does
1345// double the read time, but is still faster than adding points to waves as the file is read
1346//     
1347
1348        tic()
1349
1350        Open/R fileref as filepathstr
1351        do
1352                do
1353                        FReadLine fileref, buffer                       //skip the "blank" lines that have one character
1354                while(strlen(buffer) == 1)             
1355
1356                if (strlen(buffer) == 0)
1357                        break
1358                endif
1359               
1360                sscanf buffer,"%x",dataval
1361               
1362                // two most sig bits (31-30)
1363                type = (dataval & 0xC0000000)/1073741824                //right shift by 2^30
1364                               
1365                if(type == 0)
1366                        num0 += 1
1367                        numXYevents += 1
1368                endif
1369                if(type == 2)
1370                        num2 += 1
1371                        numXYevents += 1
1372                endif
1373                if(type == 1)
1374                        num1 += 1
1375                endif
1376                if(type == 3)
1377                        num3 += 1
1378                endif   
1379               
1380                bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1381               
1382                if(type==0 || type==2)
1383                        numPP += round(bit29)
1384                endif
1385               
1386                if(type==1 || type==3)
1387                        numT0 += round(bit29)
1388                endif
1389               
1390                if(dataval == 0)
1391                        numZero += 1
1392                endif
1393               
1394        while(1)
1395        Close fileref
1396//              done counting the number of XY events
1397        printf("Igor pre-scan done in  ")
1398        toc()
1399       
1400
1401        Print "(Igor) numT0 = ",numT0   
1402        Print "num0 = ",num0   
1403        Print "num1 = ",num1   
1404        Print "num2 = ",num2   
1405        Print "num3 = ",num3   
1406
1407//
1408//     
1409//      Printf "numXYevents = %d\r",numXYevents
1410//      Printf "XY = num0 = %d\r",num0
1411//      Printf "XY time = num2 = %d\r",num2
1412//      Printf "time MSW = num1 = %d\r",num1
1413//      Printf "Rollover = num3 = %d\r",num3
1414//      Printf "num0 + num2 = %d\r",num0+num2
1415
1416// dispStr will be displayed on the panel
1417        fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
1418       
1419        sprintf tmpStr, "%s: %d total bytes\r",fileStr,totBytes
1420        dispStr = tmpStr
1421        sprintf tmpStr,"numXYevents = %d\r",numXYevents
1422        dispStr += tmpStr
1423        sprintf tmpStr,"PP = %d  :  ",numPP
1424        dispStr += tmpStr
1425        sprintf tmpStr,"ZeroData = %d\r",numZero
1426        dispStr += tmpStr
1427        sprintf tmpStr,"Rollover = %d",num3
1428        dispStr += tmpStr
1429
1430        // /I/U is unsigned 32-bit integer (for the time)
1431        // /B/U is unsigned 8-bit integer (max val=255) for the x and y values
1432       
1433        Make/O/U/N=(numXYevents) xLoc,yLoc
1434        Make/O/D/N=(numXYevents) timePt
1435////    Make/O/U/N=(numXYevents/decFac) xLoc,yLoc
1436////    Make/O/D/N=(numXYevents/decFac) timePt
1437//      Make/O/U/N=(totBytes/4) xLoc,yLoc               //too large, trim when done (bad idea)
1438//      Make/O/D/N=(totBytes/4) timePt
1439        Make/O/D/N=1000 badTimePt,badEventNum,PPTime,PPEventNum,T0Time,T0EventNum
1440        badTimePt=0
1441        badEventNum=0
1442        PPTime=0
1443        PPEventNum=0
1444        T0Time=0
1445        T0EventNum=0
1446        xLoc=0
1447        yLoc=0
1448        timePt=0
1449       
1450        nRoll = 0               //number of rollover events
1451        roll_time = 2^26                //units of 10-7 sec
1452       
1453        NVAR removeBadEvents = root:Packages:NIST:Event:gRemoveBadEvents
1454       
1455        time_msw=0
1456       
1457        tic()
1458       
1459        ii = 0          //indexes the points in xLoc,yLoc,timePt
1460////    keep = decFac           //keep the first point
1461       
1462       
1463        Open/R fileref as filepathstr
1464       
1465        // remove events at the beginning up to a type==2 so that the msw and lsw times are reset properly
1466        if(RemoveBadEvents == 1)
1467                do
1468                        do
1469                                FReadLine fileref, buffer                       //skip the "blank" lines that have one character
1470                        while(strlen(buffer) == 1)             
1471       
1472                        if (strlen(buffer) == 0)
1473                                break
1474                        endif
1475                       
1476                        sscanf buffer,"%x",dataval
1477                // two most sig bits (31-30)
1478                        type = (dataval & 0xC0000000)/1073741824                //right shift by 2^30
1479                       
1480                        if(type == 2)
1481                                // this is the first event with a proper time value, so process the XY-time event as ususal
1482                                // and then break to drop to the main loop, where the next event == type 1
1483                               
1484                                xval = xMax - (dataval & 255)                                           //last 8 bits (7-0)
1485                                yval = (dataval & 65280)/256                                            //bits 15-8, right shift by 2^8
1486               
1487                                time_lsw = (dataval & 536805376)/65536                  //13 bits, 28-16, right shift by 2^16
1488               
1489                                if(verbose)
1490                //                                      printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1491                                        printf "%u : %u : %u : %u\r",dataval,timeval,xval,yval
1492                                endif
1493                               
1494                                // this is the first point, be sure that ii = 0, and always keep this point
1495////                            if(keep==decFac)
1496                                        ii = 0
1497                                        xLoc[ii] = xval
1498                                        yLoc[ii] = yval
1499////                                    keep = 0
1500////                            endif
1501                                Print "At beginning of file, numBad = ",numBad
1502                                break   // the next do loop processes the bulk of the file (** the next event == type 1 = MIR)
1503                        else
1504                                numBad += 1
1505                                numRemoved += 1
1506                        endif
1507                       
1508                        //ii+=1         don't increment the counter
1509                while(1)
1510        endif
1511       
1512        // now read the main portion of the file.
1513////    // keep is = 0 if bad points were removed, or is decFac is I need to keep the first point
1514        do
1515                do
1516                        FReadLine fileref, buffer                       //skip the "blank" lines that have one character
1517                while(strlen(buffer) == 1)             
1518
1519                if (strlen(buffer) == 0)                                // this marks the end of the file and is our only way out
1520                        break
1521                endif
1522               
1523                sscanf buffer,"%x",dataval
1524               
1525
1526//              type = (dataval & ~(2^32 - 2^30 -1))/2^30
1527
1528                // two most sig bits (31-30)
1529                type = (dataval & 0xC0000000)/1073741824                //right shift by 2^30
1530               
1531                //
1532                // The defintions of the event types
1533                //
1534                //Constant ATXY = 0
1535                //Constant ATXYM = 2
1536                //Constant ATMIR = 1
1537                //Constant ATMAR = 3
1538                //
1539                                               
1540                if(verbose > 0)
1541                        verbose -= 1
1542                endif
1543//             
1544                switch(type)
1545                        case ATXY:              // 0
1546                                if(verbose)             
1547                                        printf "XY : "         
1548                                endif
1549                               
1550                                // if the datavalue is == 0, just skip it now (it can only be interpreted as type 0, obviously)
1551                                if(dataval == 0 && RemoveBadEvents == 1)
1552                                        numRemoved += 1
1553//                                      Print "zero at ii= ",ii
1554                                        break           //don't increment ii
1555                                endif
1556                               
1557                                // if it's a pileup event, skip it now (this can be either type 0 or 2)
1558                                bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1559                                if(bit29 == 1 && RemoveBadEvents == 1)
1560                                        PPTime[tmpPP] = timeval
1561                                        PPEventNum[tmpPP] = ii
1562                                        tmpPP += 1
1563                                        numRemoved += 1
1564                                        break           //don't increment ii
1565                                endif
1566                               
1567//                              xval = ~(dataval & ~(2^32 - 2^8)) & 127
1568//                              yval = ((dataval & ~(2^32 - 2^16 ))/2^8) & 127
1569//                              time_lsw = (dataval & ~(2^32 - 2^29))/2^16
1570
1571                                xval = xMax - (dataval & 255)                                           //last 8 bits (7-0)
1572                                yval = (dataval & 65280)/256                                            //bits 15-8, right shift by 2^8
1573                                time_lsw = (dataval & 536805376)/65536                  //13 bits, 28-16, right shift by 2^16
1574
1575                                timeval = trunc( nRoll*roll_time + (time_msw * (8192)) + time_lsw )             //left shift msw by 2^13, then add in lsw, as an integer
1576                                if (timeval > t_longest)
1577                                        t_longest = timeval
1578                                endif
1579                               
1580                               
1581                                // catch the "bad" events:
1582                                // if an XY event follows a rollover, time_msw is 0 by definition, but does not immediately get
1583                                // re-evalulated here. Throw out only the immediately following points where msw is still 8191
1584                                if(rolloverHappened && RemoveBadEvents == 1)
1585                                        // maybe a bad event
1586                                        if(time_msw == 8191)
1587                                                badTimePt[numBad] = timeVal
1588                                                badEventNum[numBad] = ii
1589                                                numBad +=1
1590                                                numRemoved += 1
1591                                        else
1592                                                // time_msw has been reset, points are good now, so keep this one
1593////                                            if(keep==decFac)
1594                                                        xLoc[ii] = xval
1595                                                        yLoc[ii] = yval
1596                                                        timePt[ii] = timeval
1597                                                       
1598        //                                              if(xval == 127 && yval == 0)
1599        //                                                      // check bit 29
1600        //                                                      bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1601        //                                                      Print "XY=127,0 : bit29 = ",bit29
1602        //                                              endif
1603                                                       
1604                                                        ii+=1
1605                                                        rolloverHappened = 0
1606////                                                    keep = 0
1607////                                            else
1608////                                                    keep += 1
1609////                                            endif
1610                                        endif
1611                                else
1612                                        // normal processing of good point, keep it
1613////                                    if(keep==decFac)
1614                                                xLoc[ii] = xval
1615                                                yLoc[ii] = yval
1616                                                timePt[ii] = timeval
1617                                       
1618        //                                      if(xval == 127 && yval == 0)
1619        //                                              // check bit 29
1620        //                                              bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1621        //                                              Printf "XY=127,0 : bit29 = %u : d=%u\r",bit29,dataval
1622        //                                      endif
1623                                                ii+=1
1624////                                            keep = 0
1625////                                    else
1626////                                            keep += 1
1627////                                    endif
1628                                endif
1629
1630
1631                                if(verbose)             
1632//                                      printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1633                                        printf "d=%u : t=%u : msw=%u : lsw=%u : %u : %u \r",dataval,timeval,time_msw,time_lsw,xval,yval
1634                                endif                           
1635       
1636//                              verbose = 0
1637                                break
1638                        case ATXYM: // 2
1639                                if(verbose)
1640                                        printf "XYM : "
1641                                endif
1642                               
1643                                // if it's a pileup event, skip it now (this can be either type 0 or 2)
1644                                // - but can I do this if this is an XY-time event? This will lead to a wrong time, and a time
1645                                // assigned to an XY (0,0)...
1646//                              bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1647//                              if(bit29 == 1 && RemoveBadEvents == 1)
1648//                                      Print "*****Bit 29 (PP) event set for Type==2, but not handled, ii = ",ii
1649////                                    break           //don't increment ii
1650//                              endif
1651                               
1652//                              xval = ~(dataval & ~(2^32 - 2^8)) & 127
1653//                              yval = ((dataval & ~(2^32 - 2^16 ))/2^8) & 127
1654//                              time_lsw =  (dataval & ~(2^32 - 2^29 ))/2^16            //this method gives a FP result!! likely since the "^" operation gives FP result...
1655
1656                                xval = xMax - (dataval & 255)                                           //last 8 bits (7-0)
1657                                yval = (dataval & 65280)/256                                            //bits 15-8, right shift by 2^8
1658
1659                                time_lsw = (dataval & 536805376)/65536                  //13 bits, 28-16, right shift by 2^16 (result is integer)
1660
1661                                if(verbose)
1662//                                      printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1663                                        printf "%u : %u : %u : %u\r",dataval,timeval,xval,yval
1664                                endif
1665                               
1666////                            if(keep==decFac)                        //don't reset keep yet, do this only when ii increments
1667                                        xLoc[ii] = xval
1668                                        yLoc[ii] = yval
1669////                            endif
1670                               
1671                                // don't fill in the time yet, or increment the index ii
1672                                // the next event MUST be ATMIR with the MSW time bits
1673                                //
1674//                              verbose = 0
1675                                break
1676                        case ATMIR:  // 1
1677                                if(verbose)
1678                                        printf "MIR : "
1679                                endif
1680
1681                                time_msw =  (dataval & 536805376)/65536                 //13 bits, 28-16, right shift by 2^16
1682                                timeval = trunc( nRoll*roll_time + (time_msw * (8192)) + time_lsw )
1683                                if (timeval > t_longest)
1684                                        t_longest = timeval
1685                                endif
1686                                if(verbose)
1687//                                      printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1688                                        printf "d=%u : t=%u : msw=%u : lsw=%u : tlong=%u\r",dataval,timeval,time_msw,time_lsw,t_longest
1689                                endif
1690                               
1691                                // the XY position was in the previous event ATXYM
1692////                            if(keep == decFac)
1693                                        timePt[ii] = timeval
1694////                            endif
1695
1696                                bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1697                                if(bit29 != 0)          // bit 29 set is a T0 event
1698                                        //Printf "bit29 = 1 at ii = %d : type = %d\r",ii,type
1699                                        T0Time[tmpT0] = timeval
1700                                        T0EventNum[tmpT0] = ii
1701                                        tmpT0 += 1
1702                                        // reset nRoll = 0 for calcluating the time
1703                                        nRoll = 0
1704                                endif
1705                               
1706////                            if(keep == decFac)                     
1707                                        ii+=1
1708////                                    keep = 0
1709////                            endif
1710//                              verbose = 0
1711                                break
1712                        case ATMAR:  // 3
1713                                if(verbose3)
1714//                                      verbose = 15
1715//                                      verbose = 2
1716                                        printf "MAR : "
1717                                endif
1718                               
1719                                // do something with the rollover event?
1720                               
1721                                // check bit 29
1722                                bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
1723                                nRoll += 1
1724// not doing anything with these bits yet       
1725                                bit28 = (dataval & 0x10000000)/268435456                //bit 28 only, shift by 2^28   
1726                                bit27 = (dataval & 0x08000000)/134217728        //bit 27 only, shift by 2^27
1727
1728                                if(verbose3)
1729                                        printf "d=%u : b29=%u : b28=%u : b27=%u : #Roll=%u \r",dataval,bit29, bit28, bit27,nRoll
1730                                endif
1731                               
1732                                if(bit29 != 0)          // bit 29 set is a T0 event
1733                                        //Printf "bit29 = 1 at ii = %d : type = %d\r",ii,type
1734                                        T0Time[tmpT0] = timeval
1735                                        T0EventNum[tmpT0] = ii
1736                                        tmpT0 += 1
1737                                        // reset nRoll = 0 for calcluating the time
1738                                        nRoll = 0
1739                                endif
1740                               
1741                                rolloverHappened = 1
1742
1743                                break
1744                endswitch
1745               
1746//              if(ii<18)
1747//                      printf "TYPE=%d : ii=%d : d=%u : t=%u : msw=%u : lsw=%u : %u : %u \r",type,ii,dataval,timeval,time_msw,time_lsw,xval,yval
1748//              endif   
1749                       
1750        while(1)
1751       
1752        Close fileref
1753       
1754        printf("Igor full file read done in  ")
1755        toc()
1756       
1757        Print "Events removed (Igor) = ",numRemoved
1758       
1759        sPrintf tmpStr,"\rBad Rollover Events = %d (%4.4g %% of events)",numBad,numBad/numXYevents*100
1760        dispStr += tmpStr
1761        sPrintf tmpStr,"\rTotal Events Removed = %d (%4.4g %% of events)",numRemoved,numRemoved/numXYevents*100
1762        dispStr += tmpStr
1763        SetDataFolder root:
1764       
1765        return(0)
1766       
1767End
1768
1769//////////////
1770//
1771// This calls the XOP, as an operation to load the events
1772//
1773// -- it's about 35x faster than the Igor code, so I guess that's OK.
1774//
1775// conditional compile the whole inner workings in case XOP is not present
1776Function LoadEvents_XOP()
1777#if (exists("EventLoadWave")==4)
1778       
1779//      NVAR time_msw = root:Packages:NIST:Event:gEvent_time_msw
1780//      NVAR time_lsw = root:Packages:NIST:Event:gEvent_time_lsw
1781        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1782       
1783        SVAR filepathstr = root:Packages:NIST:Event:gEvent_logfile
1784        SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
1785       
1786        SetDataFolder root:Packages:NIST:Event
1787
1788
1789
1790        Variable fileref
1791        String buffer
1792        String fileStr,tmpStr
1793        Variable dataval,timeval,type,numLines,verbose,verbose3
1794        Variable xval,yval,rollBit,nRoll,roll_time,bit29,bit28,bit27
1795        Variable ii,flaggedEvent,rolloverHappened,numBad=0,tmpPP=0,tmpT0=0
1796        Variable Xmax, yMax
1797       
1798        xMax = 127              // number the detector from 0->127
1799        yMax = 127
1800       
1801        numLines = 0
1802
1803        //Have to declare local variables for Loadwave so that this compiles without XOP.
1804        String S_waveNames
1805        //  and those for the XOP
1806        Variable V_nXYevents,V_num1,V_num2,V_num3,V_num0,V_totBytes,V_numPP,V_numT0,V_numDL,V_numFF,V_numZero
1807        Variable V_numBad,V_numRemoved
1808       
1809        // what I really need is the number of XY events
1810        Variable numXYevents,num1,num2,num3,num0,totBytes,numPP,numT0,numDL,numFF,numZero
1811        Variable numRemoved
1812        numXYevents = 0
1813        num0 = 0
1814        num1 = 0
1815        num2 = 0
1816        num3 = 0
1817        numPP = 0
1818        numT0 = 0
1819        numDL = 0
1820        numFF = 0
1821        numZero = 0
1822        numRemoved = 0
1823
1824// get the total number of bytes in the file
1825        Open/R fileref as filepathstr
1826                FStatus fileref
1827        Close fileref
1828
1829        totBytes = V_logEOF
1830        Print "total bytes = ", totBytes
1831       
1832        if(EventDataType(filePathStr) != 1)             //if not hst extension of raw data
1833                Abort "Data must be raw event data with file extension .hst for this operation"
1834                return(0)
1835        endif
1836//
1837//      Print "scan only"
1838//      tic()
1839//              EventLoadWave/R/N=EventWave/W filepathstr
1840//      toc()
1841
1842////
1843//
1844//  use the XOP operation to load in the data
1845// -- this does everything - the pre-scan and creating the waves
1846//
1847// need to zero the waves before loading, just in case
1848//
1849
1850        NVAR removeBadEvents = root:Packages:NIST:Event:gRemoveBadEvents
1851
1852tic()
1853
1854//      Wave/Z wave0=wave0
1855//      Wave/Z wave1=wave1
1856//      Wave/Z wave2=wave2
1857//
1858//      if(WaveExists(wave0))
1859//              MultiThread wave0=0
1860//      endif
1861//      if(WaveExists(wave1))
1862//              MultiThread wave1=0
1863//      endif
1864//      if(WaveExists(wave2))
1865//              MultiThread wave2=0
1866//      endif
1867
1868        if(removeBadEvents)
1869                EventLoadWave/R/N=EventWave filepathstr
1870        else
1871                EventLoadWave/N=EventWave  filepathstr
1872        endif
1873
1874
1875        Print "XOP files loaded = ",S_waveNames
1876
1877////            -- copy the waves over to xLoc,yLoc,timePt
1878        Wave/Z EventWave0=EventWave0
1879        Wave/Z EventWave1=EventWave1
1880        Wave/Z EventWave2=EventWave2
1881       
1882       
1883        Duplicate/O EventWave0,xLoc
1884        KillWaves/Z EventWave0
1885
1886        Duplicate/O EventWave1,yLoc
1887        KillWaves/Z EventWave1
1888
1889        Duplicate/O EventWave2,timePt
1890        KillWaves/Z EventWave2
1891
1892// could do this, but rescaled time will neeed to be converted to SP (or DP)
1893// and Igor loader was written with Make generating SP/DP waves
1894        // /I/U is unsigned 32-bit integer (for the time)
1895        // /B/U is unsigned 8-bit integer (max val=255) for the x and y values
1896       
1897//      Redimension/B/U xLoc,yLoc
1898//      Redimension/I/U timePt
1899
1900        // access the variables from the XOP
1901        numT0 = V_numT0
1902        numPP = V_numPP
1903        num0 = V_num0
1904        num1 = V_num1
1905        num2 = V_num2
1906        num3 = V_num3
1907        numXYevents = V_nXYevents
1908        numZero = V_numZero
1909        numBad = V_numBad
1910        numRemoved = V_numRemoved
1911       
1912        Print "(XOP) numT0 = ",numT0   
1913        Print "num0 = ",num0   
1914        Print "num1 = ",num1   
1915        Print "num2 = ",num2   
1916        Print "num3 = ",num3   
1917       
1918
1919// dispStr will be displayed on the panel
1920        fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
1921       
1922        sprintf tmpStr, "%s: %d total bytes\r",fileStr,totBytes
1923        dispStr = tmpStr
1924        sprintf tmpStr,"numXYevents = %d\r",numXYevents
1925        dispStr += tmpStr
1926        sprintf tmpStr,"PP = %d  :  ",numPP
1927        dispStr += tmpStr
1928        sprintf tmpStr,"ZeroData = %d\r",numZero
1929        dispStr += tmpStr
1930        sprintf tmpStr,"Rollover = %d",num3
1931        dispStr += tmpStr
1932
1933        toc()
1934       
1935        Print "Events removed (XOP) = ",numRemoved
1936       
1937        sPrintf tmpStr,"\rBad Rollover Events = %d (%4.4g %% of events)",numBad,numBad/numXYevents*100
1938        dispStr += tmpStr
1939        sPrintf tmpStr,"\rTotal Events Removed = %d (%4.4g %% of events)",numRemoved,numRemoved/numXYevents*100
1940        dispStr += tmpStr
1941
1942
1943// simply to compile a table of # XY vs # bytes
1944//      Wave/Z nxy = root:numberXY
1945//      Wave/Z nBytes = root:numberBytes
1946//      if(WaveExists(nxy) && WaveExists(nBytes))
1947//              InsertPoints 0, 1, nxy,nBytes
1948//              nxy[0] = numXYevents
1949//              nBytes[0] = totBytes
1950//      endif
1951
1952        SetDataFolder root:
1953
1954#endif 
1955        return(0)
1956       
1957End
1958
1959//////////////
1960
1961Proc BinEventBarGraph()
1962       
1963        DoWindow/F EventBarGraph
1964        if(V_flag == 0)
1965                PauseUpdate; Silent 1           // building window...
1966                String fldrSav0= GetDataFolder(1)
1967                SetDataFolder root:Packages:NIST:Event:
1968                Display /W=(110,705,610,1132)/N=EventBarGraph /K=1 binCount vs binEndTime
1969                SetDataFolder fldrSav0
1970                ModifyGraph mode=5
1971                ModifyGraph marker=19
1972                ModifyGraph lSize=2
1973                ModifyGraph rgb=(0,0,0)
1974                ModifyGraph msize=2
1975                ModifyGraph hbFill=2
1976                ModifyGraph gaps=0
1977                ModifyGraph usePlusRGB=1
1978                ModifyGraph toMode=1
1979                ModifyGraph useBarStrokeRGB=1
1980                ModifyGraph standoff=0
1981                SetAxis left 0,*
1982                Label bottom "\\Z14Time (seconds)"
1983                Label left "\\Z14Number of Events"
1984        endif
1985End
1986
1987
1988Proc ShowBinTable()
1989
1990        DoWindow/F BinEventTable
1991        if(V_flag == 0)
1992                PauseUpdate; Silent 1           // building window...
1993                String fldrSav0= GetDataFolder(1)
1994                SetDataFolder root:Packages:NIST:Event:
1995                Edit/W=(498,699,1003,955) /K=1/N=BinEventTable binCount,binEndTime,timeWidth
1996                ModifyTable format(Point)=1,sigDigits(binEndTime)=8,width(binEndTime)=100
1997                SetDataFolder fldrSav0
1998        endif
1999EndMacro
2000
2001
2002// only show the first 1500 data points
2003//
2004Proc ShowRescaledTimeGraph()
2005
2006        DoWindow/F RescaledTimeGraph
2007        if(V_flag == 0)
2008                PauseUpdate; Silent 1           // building window...
2009                String fldrSav0= GetDataFolder(1)
2010                SetDataFolder root:Packages:NIST:Event:
2011                Display /W=(25,44,486,356)/K=1/N=RescaledTimeGraph rescaledTime
2012                SetDataFolder fldrSav0
2013                ModifyGraph mode=0
2014                ModifyGraph marker=19
2015                ModifyGraph rgb(rescaledTime)=(0,0,0)
2016                ModifyGraph msize=1
2017//              SetAxis/A=2 left                        //only autoscale the visible data (based on the bottom limits)
2018                SetAxis bottom 0,1500
2019                ErrorBars rescaledTime OFF
2020                Label left "\\Z14Time (seconds)"
2021                Label bottom "\\Z14Event number"
2022                ShowInfo
2023        endif
2024       
2025EndMacro
2026
2027
2028
2029Proc ExportSlicesAsVAX(firstNum,prefix)
2030        Variable firstNum=1
2031        String prefix="SAMPL"
2032
2033        SaveSlicesAsVAX(firstNum,prefix[0,4])           //make sure that the prefix is 5 chars
2034End
2035
2036//////// procedures to be able to export the slices as RAW VAX files.
2037//
2038// 1- load the raw data file to use the header (it must already be in RAW)
2039// 1.5- copy the raw data to the temp folder (STO)
2040// 1.7- ask for the prefix and starting run number (these are passed in)
2041// 2- copy the slice of data to the temp folder (STO)
2042// 3- touch up the time/counts in the slice header values in STO
2043// 4- write out the VAX file
2044// 5- repeat (2-4) for the number of slices
2045//
2046//
2047Function SaveSlicesAsVAX(firstNum,prefix)
2048        Variable firstNum
2049        String prefix
2050
2051        DoAlert 1,"Is the full data file loaded as a RAW data file? If not, load it and start over..."
2052        if(V_flag == 2)
2053                return (0)
2054        endif
2055       
2056// copy the contents of RAW to STO so I can work from there
2057        CopyWorkContents("RAW","STO")
2058
2059        // now declare all of the waves, now that they are sure to be there
2060
2061        WAVE slicedData=root:Packages:NIST:Event:slicedData
2062        Make/O/D/N=(128,128) curSlice
2063       
2064        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
2065        WAVE binEndTime = root:Packages:NIST:Event:binEndTime
2066
2067        Wave rw=root:Packages:NIST:STO:realsRead
2068        Wave iw=root:Packages:NIST:STO:integersRead
2069        Wave/T tw=root:Packages:NIST:STO:textRead
2070        Wave data=root:Packages:NIST:STO:data
2071        Wave linear_data=root:Packages:NIST:STO:linear_data
2072       
2073       
2074        Wave rw_raw=root:Packages:NIST:RAW:realsRead
2075        Wave iw_raw=root:Packages:NIST:RAW:integersRead
2076        Wave/T tw_raw=root:Packages:NIST:RAW:textRead
2077
2078// for generating the alphanumeric
2079        String timeStr= secs2date(datetime,-1)
2080        String monthStr=StringFromList(1, timeStr  ,"/")
2081        String numStr="",labelStr
2082
2083        Variable ii,err,binFraction
2084       
2085        for(ii=0;ii<nslices;ii+=1)
2086
2087                //get the current slice and put it in the STO folder
2088                curSlice = slicedData[p][q][ii]
2089                data = curSlice
2090                linear_data = curSlice
2091               
2092                // touch up the header as needed
2093                // count time = iw[2]
2094                // monCt = rw[0]
2095                // detCt = rw[2]
2096                //tw[0] must now be the file name
2097                //
2098                // count time = fraction of total binning * total count time
2099                binFraction = (binEndTime[ii+1]-binEndTime[ii])/(binEndTime[nslices]-binEndTime[0])
2100               
2101                iw[2] = trunc(binFraction*iw_raw[2])
2102                rw[0] = trunc(binFraction*rw_raw[0])
2103                rw[2] = sum(curSlice,-inf,inf)          //total counts in slice
2104       
2105                if(firstNum<10)
2106                        numStr = "00"+num2str(firstNum)
2107                else
2108                        if(firstNum<100)
2109                                numStr = "0"+num2str(firstNum)
2110                        else
2111                                numStr = num2str(firstNum)
2112                        Endif
2113                Endif   
2114                tw[0] = prefix+numstr+".SA2_EVE_"+(num2char(str2num(monthStr)+64))+numStr
2115                labelStr = tw_raw[6]
2116               
2117                labelStr = PadString(labelStr,60,0x20)  //60 fortran-style spaces
2118                tw[6] = labelStr[0,59]
2119               
2120                //write out the file - this uses the tw[0] and home path
2121                Write_VAXRaw_Data("STO","",0)
2122
2123                //increment the run number, alpha
2124                firstNum += 1   
2125        endfor
2126
2127        return(0)
2128End
2129
2130
2131
2132
2133
2134/////////////
2135//The histogramming
2136//
2137// 6 AUG 2012
2138//
2139// from Igor Exchange, RGerkin
2140//  http://www.igorexchange.com/node/1373
2141// -- see the related thread on the mailing list
2142//
2143//Function Setup_JointHistogram()
2144//
2145////    tic()
2146//
2147//      make/D /o/n=1000000 data1=gnoise(1), data2=gnoise(1)
2148//      make/D /o/n=(25,25) myHist
2149//      setscale x,-3,3,myHist
2150//      setscale y,-3,3,myHist
2151//      IndexForHistogram(data1,data2,myhist)
2152//      Wave index=SavedIndex
2153//      EV_JointHistogram(data1,data2,myHist,index)
2154//      NewImage myHist
2155//     
2156////    toc()
2157//     
2158//End
2159
2160
2161Function EV_JointHistogram(w0,w1,hist,index)
2162        wave w0,w1,hist,index
2163 
2164        variable bins0=dimsize(hist,0)
2165        variable bins1=dimsize(hist,1)
2166        variable n=numpnts(w0)
2167        variable left0=dimoffset(hist,0)
2168        variable left1=dimoffset(hist,1)
2169        variable right0=left0+bins0*dimdelta(hist,0)
2170        variable right1=left1+bins1*dimdelta(hist,1)
2171       
2172        // Compute the histogram and redimension it. 
2173        histogram /b={0,1,bins0*bins1} index,hist
2174        redimension/D /n=(bins0,bins1) hist // Redimension to 2D. 
2175        setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension. 
2176        setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension. 
2177End
2178
2179
2180// histogram with a point range
2181//
2182// x- just need to send x2pnt or findLevel, or something similar to define the POINT
2183// values
2184//
2185// x- can also speed this up since the index only needs to be done once, so the
2186// histogram operation can be done separately, as the bins require
2187//
2188//
2189Function JointHistogramWithRange(w0,w1,hist,index,pt1,pt2)
2190        wave w0,w1,hist,index
2191        Variable pt1,pt2
2192 
2193        variable bins0=dimsize(hist,0)
2194        variable bins1=dimsize(hist,1)
2195        variable n=numpnts(w0)
2196        variable left0=dimoffset(hist,0)
2197        variable left1=dimoffset(hist,1)
2198        variable right0=left0+bins0*dimdelta(hist,0)
2199        variable right1=left1+bins1*dimdelta(hist,1)
2200
2201        // Compute the histogram and redimension it. 
2202        histogram /b={0,1,bins0*bins1}/R=[pt1,pt2] index,hist
2203        redimension/D /n=(bins0,bins1) hist // Redimension to 2D. 
2204        setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension. 
2205        setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension. 
2206End
2207
2208
2209// just does the indexing, creates wave SavedIndex in the current folder for the index
2210//
2211Function IndexForHistogram(w0,w1,hist)
2212        wave w0,w1,hist
2213 
2214        variable bins0=dimsize(hist,0)
2215        variable bins1=dimsize(hist,1)
2216        variable n=numpnts(w0)
2217        variable left0=dimoffset(hist,0)
2218        variable left1=dimoffset(hist,1)
2219        variable right0=left0+bins0*dimdelta(hist,0)
2220        variable right1=left1+bins1*dimdelta(hist,1)
2221 
2222        // Scale between 0 and the number of bins to create an index wave. 
2223        if(ThreadProcessorCount<4) // For older machines, matrixop is faster. 
2224                matrixop /free idx=round(bins0*(w0-left0)/(right0-left0))+bins0*round(bins1*(w1-left1)/(right1-left1))
2225        else // For newer machines with many cores, multithreading with make is faster. 
2226                make/free/n=(n) idx
2227                multithread idx=round(bins0*(w0-left0)/(right0-left0))+bins0*round(bins1*(w1-left1)/(right1-left1))
2228        endif
2229 
2230        KillWaves/Z SavedIndex
2231        MoveWave idx,SavedIndex
2232       
2233//      // Compute the histogram and redimension it. 
2234//      histogram /b={0,1,bins0*bins1} idx,hist
2235//      redimension /n=(bins0,bins1) hist // Redimension to 2D. 
2236//      setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension. 
2237//      setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension. 
2238End
2239
2240
2241
2242
2243
2244////////////// Post-processing of the event mode data
2245Proc ShowEventCorrectionPanel()
2246        DoWindow/F EventCorrectionPanel
2247        if(V_flag ==0)
2248                EventCorrectionPanel()
2249        EndIf
2250End
2251
2252Proc EventCorrectionPanel()
2253
2254        PauseUpdate; Silent 1           // building window...
2255        SetDataFolder root:Packages:NIST:Event:
2256       
2257        if(exists("rescaledTime") == 1)
2258                Display /W=(35,44,761,533)/K=2 rescaledTime
2259                DoWindow/C EventCorrectionPanel
2260                ModifyGraph mode=0
2261                ModifyGraph marker=19
2262                ModifyGraph rgb=(0,0,0)
2263                ModifyGraph msize=1
2264                ErrorBars rescaledTime OFF
2265                Label left "\\Z14Time (seconds)"
2266                Label bottom "\\Z14Event number"       
2267                SetAxis bottom 0,0.10*numpnts(rescaledTime)             //show 1st 10% of data for speed in displaying
2268               
2269                ControlBar 100
2270                Button button0,pos={18,12},size={70,20},proc=EC_AddCursorButtonProc,title="Cursors"
2271                Button button1,pos={18,38},size={80,20},proc=EC_ShowAllButtonProc,title="All Data"
2272                Button button2,pos={18,64},size={110,20},proc=EC_DoDifferential,title="Differential"
2273
2274
2275                SetVariable setvar0 pos={153,12},title="Tol",size={90,20},value=gStepTolerance
2276                Button button3,pos={153,38},size={100,20},proc=EC_AutoCorrectSteps,title="Auto Correct"
2277                Button button4,pos={153,64},size={110,20},proc=EC_AddFindNext,title="Add Find Next"
2278
2279               
2280                Button button5,pos={285,12},size={80,20},proc=EC_AddTimeButtonProc,title="Add 6.7s"
2281                Button button6,pos={285,38},size={80,20},proc=EC_SubtractTimeButtonProc,title="Subtr 6.7s"             
2282                Button button7,pos={285,64},size={90,20},proc=EC_TrimPointsButtonProc,title="Trim points"
2283               
2284       
2285                Button button8,pos={400,12},size={110,20},proc=EC_FindStepButton_down,title="Find Step Down"
2286                Button button9,pos={400,38},size={110,20},proc=EC_FindStepButton_up,title="Find Step Up"
2287                Button button10,pos={400,64},size={100,20},proc=EC_FindOutlierButton,title="Find Outlier"
2288                Button button11,pos={520,12},size={20,20},proc=EC_NudgeCursor,title=">"
2289               
2290
2291                Button button12,pos={683,12},size={30,20},proc=EC_HelpButtonProc,title="?"
2292                Button button13,pos={625,38},size={90,20},proc=EC_SaveWavesButtonProc,title="Save Waves"
2293                Button button14,pos={655,64},size={60,20},proc=EC_DoneButtonProc,title="Done"
2294       
2295
2296        else
2297                DoAlert 0, "Please load some event data, then you'll have something to edit."
2298        endif
2299       
2300        SetDataFolder root:
2301       
2302EndMacro
2303
2304
2305Function EC_AutoCorrectSteps(ba)
2306        STRUCT WMButtonAction &ba
2307
2308        switch( ba.eventCode )
2309                case 2: // mouse up
2310                        // click code here
2311                        AutoFix_Rollover_Steps()
2312                       
2313                        break
2314                case -1: // control being killed
2315                        break
2316        endswitch
2317               
2318        return(0)
2319End
2320
2321Function EC_NudgeCursor(ba)
2322        STRUCT WMButtonAction &ba
2323
2324        switch( ba.eventCode )
2325                case 2: // mouse up
2326                        // click code here
2327                        // nudge the leftmost cursor to the right two points
2328                        // presumably to bypass a desired time step
2329                        //
2330       
2331                        Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
2332                        Variable ptA,ptB,lo
2333                        ptA = pcsr(A)
2334                        ptB = pcsr(B)
2335                        lo=min(ptA,ptB)
2336
2337                        if(lo == ptA)
2338                                Cursor/P A rescaledTime ptA+2   //at the point + 2
2339                                Print "Nudged cursor A two points right"
2340                        else
2341                                Cursor/P B rescaledTime ptB+2   //at the point + 2
2342                                Print "Nudged cursor B two points right"
2343                        endif
2344                       
2345                        break
2346                case -1: // control being killed
2347                        break
2348        endswitch
2349               
2350        return(0)
2351End
2352
2353Function EC_AddFindNext(ba)
2354        STRUCT WMButtonAction &ba
2355
2356        switch( ba.eventCode )
2357                case 2: // mouse up
2358                        // click code here
2359                        // add time     
2360                        EC_AddTimeButtonProc(ba)
2361                       
2362                // re-do the differential
2363                        EC_DoDifferential("")
2364                       
2365                // find the next step down
2366                        EC_FindStepButton_down("")
2367                       
2368                        break
2369                case -1: // control being killed
2370                        break
2371        endswitch
2372               
2373        return(0)
2374End
2375
2376
2377
2378Function EC_AddCursorButtonProc(ba) : ButtonControl
2379        STRUCT WMButtonAction &ba
2380
2381        switch( ba.eventCode )
2382                case 2: // mouse up
2383                        // click code here
2384                        SetDataFolder root:Packages:NIST:Event:
2385                       
2386                        Wave rescaledTime = rescaledTime
2387                        Cursor/P A rescaledTime 0
2388                        Cursor/P B rescaledTime numpnts(rescaledTime)-1
2389                        ShowInfo
2390                        SetDataFolder root:
2391                        break
2392                case -1: // control being killed
2393                        break
2394        endswitch
2395
2396        return 0
2397End
2398
2399// updates the longest time (as does every operation of adjusting the data)
2400//
2401Function EC_AddTimeButtonProc(ba) : ButtonControl
2402        STRUCT WMButtonAction &ba
2403
2404        switch( ba.eventCode )
2405                case 2: // mouse up
2406                        // click code here
2407                        SetDataFolder root:Packages:NIST:Event:
2408                       
2409                        Wave rescaledTime = rescaledTime
2410                        Wave timePt = timePt
2411                        Variable rollTime,rollTicks,ptA,ptB,lo,hi
2412                       
2413                        rollTicks = 2^26                                // in ticks
2414                        rollTime = 2^26*1e-7            // in seconds
2415                        ptA = pcsr(A)
2416                        ptB = pcsr(B)
2417                        lo=min(ptA,ptB)
2418                        hi=max(ptA,ptB)
2419
2420                        MultiThread timePt[lo,hi] += rollTicks
2421                        MultiThread rescaledTime[lo,hi] += rollTime
2422
2423                        // updates the longest time (as does every operation of adjusting the data)
2424                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2425                        t_longest = waveMax(rescaledTime)
2426                       
2427                        SetDataFolder root:
2428                        break
2429                case -1: // control being killed
2430                        break
2431        endswitch
2432
2433        return 0
2434End
2435
2436Function EC_SubtractTimeButtonProc(ba) : ButtonControl
2437        STRUCT WMButtonAction &ba
2438
2439        switch( ba.eventCode )
2440                case 2: // mouse up
2441                        // click code here
2442                        SetDataFolder root:Packages:NIST:Event:
2443                       
2444                        Wave rescaledTime = rescaledTime
2445                        Wave timePt = timePt
2446                        Variable rollTime,rollTicks,ptA,ptB,lo,hi
2447                       
2448                        rollTicks = 2^26                                // in ticks
2449                        rollTime = 2^26*1e-7            // in seconds
2450                        ptA = pcsr(A)
2451                        ptB = pcsr(B)
2452                        lo=min(ptA,ptB)
2453                        hi=max(ptA,ptB)
2454                       
2455                        MultiThread timePt[lo,hi] -= rollTicks
2456                        MultiThread rescaledTime[lo,hi] -= rollTime
2457
2458                        // updates the longest time (as does every operation of adjusting the data)
2459                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2460                        t_longest = waveMax(rescaledTime)
2461                       
2462                        SetDataFolder root:
2463                       
2464                        break
2465                case -1: // control being killed
2466                        break
2467        endswitch
2468
2469        return 0
2470End
2471
2472// points removed are inclusive
2473//
2474// put both cursors on the same point to remove just that single point
2475//
2476Function EC_TrimPointsButtonProc(ba) : ButtonControl
2477        STRUCT WMButtonAction &ba
2478
2479        switch( ba.eventCode )
2480                case 2: // mouse up
2481                        // click code here
2482                        SetDataFolder root:Packages:NIST:Event:
2483                       
2484                        Wave rescaledTime = rescaledTime
2485                        Wave timePt = timePt
2486                        Wave xLoc = xLoc
2487                        Wave yLoc = yLoc
2488                        Variable rollTime,ptA,ptB,numElements,lo,hi
2489                       
2490                        rollTime = 2^26*1e-7            // in seconds
2491                        ptA = pcsr(A)
2492                        ptB = pcsr(B)
2493                        lo=min(ptA,ptB)
2494                        hi=max(ptA,ptB)                 
2495                        numElements = abs(ptA-ptB)+1                    //so points removed are inclusive
2496                        DeletePoints lo, numElements, rescaledTime,timePt,xLoc,yLoc
2497                       
2498                        printf "Points %g to %g have been deleted in rescaledTime, timePt, xLoc, and yLoc\r",ptA,ptB
2499                       
2500                        // updates the longest time (as does every operation of adjusting the data)
2501                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2502                        t_longest = waveMax(rescaledTime)
2503                       
2504                        SetDataFolder root:
2505                       
2506                        break
2507                case -1: // control being killed
2508                        break
2509        endswitch
2510
2511        return 0
2512End
2513
2514// un-sort the data first, then save it
2515Function EC_SaveWavesButtonProc(ba) : ButtonControl
2516        STRUCT WMButtonAction &ba
2517
2518        switch( ba.eventCode )
2519                case 2: // mouse up
2520                        // click code here
2521                       
2522//                      Execute "UndoTheSorting()"
2523                       
2524                        SetDataFolder root:Packages:NIST:Event:
2525                       
2526                        Wave rescaledTime = rescaledTime
2527                        Wave timePt = timePt
2528                        Wave xLoc = xLoc
2529                        Wave yLoc = yLoc
2530                        Save/T xLoc,yLoc,timePt,rescaledTime            //will ask for a name
2531                       
2532                        SetDataFolder root:
2533                        break
2534                case -1: // control being killed
2535                        break
2536        endswitch
2537
2538        return 0
2539End
2540
2541//
2542// this duplicates all of the bits that would be done if the "load" button was pressed
2543//
2544Function EC_ImportWavesButtonProc(ba) : ButtonControl
2545        STRUCT WMButtonAction &ba
2546
2547        switch( ba.eventCode )
2548                case 2: // mouse up
2549                        // click code here
2550                        SetDataFolder root:Packages:NIST:Event:
2551
2552                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2553                        SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
2554                        String tmpStr="",fileStr,filePathStr
2555                       
2556                        // load in the waves, saved as Igor text to preserve the data type
2557                        PathInfo/S catPathName          //should set the next dialog to the proper path...
2558                        filePathStr = PromptForPath("Pick the edited event data")
2559                        if(EventDataType(filePathStr) != 2)             //if not itx extension of edited data
2560                                Abort "Data must be edited data with file extension .itx for this operation"
2561                                return(0)
2562                        endif
2563                        LoadWave/T/O/P=catPathName filePathStr
2564//                      filePathStr = S_fileName
2565
2566                        if(strlen(S_fileName) == 0)
2567                                //user cancelled
2568                                DoAlert 0,"No file selected, nothing done."
2569                                return(0)
2570                        endif
2571                       
2572                        NVAR mode = root:Packages:NIST:Event:gEvent_Mode                                // ==0 for "stream", ==1 for Oscillatory
2573                        // clear out the old sort index, if present, since new data is being loaded
2574                        KillWaves/Z OscSortIndex
2575                        Wave timePt=timePt
2576                        Wave rescaledTime=rescaledTime
2577                       
2578                        t_longest = waveMax(rescaledTime)               //should be the last point
2579                       
2580       
2581                        fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
2582                        sprintf tmpStr, "%s: a user-modified event file\r",fileStr
2583                        dispStr = tmpStr
2584       
2585                        SetDataFolder root:
2586                        break
2587                case -1: // control being killed
2588                        break
2589        endswitch
2590
2591        return 0
2592End
2593
2594
2595Function EC_ShowAllButtonProc(ba) : ButtonControl
2596        STRUCT WMButtonAction &ba
2597
2598        switch( ba.eventCode )
2599                case 2: // mouse up
2600                        // click code here
2601                        SetAxis/A
2602                        break
2603                case -1: // control being killed
2604                        break
2605        endswitch
2606
2607        return 0
2608End
2609
2610Function EC_HelpButtonProc(ba) : ButtonControl
2611        STRUCT WMButtonAction &ba
2612
2613        switch( ba.eventCode )
2614                case 2: // mouse up
2615                        // click code here
2616                        DisplayHelpTopic/Z "Event Mode Data[Correcting for things that go wrong]"
2617                        break
2618                case -1: // control being killed
2619                        break
2620        endswitch
2621
2622        return 0
2623End
2624
2625Function EC_DoneButtonProc(ba) : ButtonControl
2626        STRUCT WMButtonAction &ba
2627
2628        switch( ba.eventCode )
2629                case 2: // mouse up
2630                        // click code here
2631                        DoWindow/K EventCorrectionPanel
2632                        break
2633                case -1: // control being killed
2634                        break
2635        endswitch
2636
2637        return 0
2638End
2639
2640//upDown 5 or -5 looks for spikes +5 or -5 std deviations from mean
2641//
2642// will search from the leftmost cursor to the end. this allows skipping of oscillations
2643// that are not timing errors. It may introduce other issues, but we'll see what happens
2644//
2645Function PutCursorsAtStep(upDown)
2646        Variable upDown
2647       
2648        SetDataFolder root:Packages:NIST:Event:
2649
2650        Wave rescaledTime=rescaledTime
2651        Wave rescaledTime_DIF=rescaledTime_DIF
2652        Variable avg,pt,zoom
2653        Variable ptA,ptB,startPt
2654               
2655        zoom = 200              //points in each direction
2656       
2657        WaveStats/M=1/Q rescaledTime_DIF
2658        avg = V_avg
2659       
2660        ptA = pcsr(A)
2661        ptB = pcsr(B)
2662        startPt = min(ptA,ptB)
2663       
2664        FindLevel/P/Q/R=[startPt] rescaledTime_DIF abs(avg)*upDown              // in case the average is negative
2665        if(V_flag==0)
2666                pt = V_levelX
2667                WaveStats/Q/R=[pt-zoom,pt+zoom] rescaledTime            // find the max/min y-values within the point range
2668        else
2669                Print "Level not found"
2670//              return(0)
2671        endif
2672       
2673        Variable loLeft,hiLeft, loBottom,hiBottom
2674        loLeft = V_min*0.98             //+/- 2%
2675        hiLeft = V_max*1.02
2676       
2677        SetAxis left loLeft,hiLeft
2678        SetAxis bottom pnt2x(rescaledTime,pt-zoom),pnt2x(rescaledTime,pt+zoom)
2679       
2680        Cursor/P A rescaledTime pt+2    //at the point
2681        Cursor/P B rescaledTime numpnts(rescaledTime)-1         //at the end
2682
2683        SetDataFolder root:
2684
2685        return(0)
2686End
2687
2688
2689// find the max (or min) of the rescaled time set
2690// and place both cursors there
2691Function fFindOutlier()
2692
2693        SetDataFolder root:Packages:NIST:Event:
2694
2695        Wave rescaledTime=rescaledTime
2696        Variable avg,pt,zoom,maxPt,minPt,maxVal,minVal
2697       
2698        zoom = 200              //points in each direction
2699       
2700        WaveStats/M=1/Q rescaledTime
2701        maxPt = V_maxLoc
2702        minPt = V_minLoc
2703        avg = V_avg
2704        maxVal = abs(V_max)
2705        minVal = abs(V_min)
2706
2707        pt = abs(maxVal - avg) > abs(minVal - avg) ? maxPt : minPt
2708       
2709//      Variable loLeft,hiLeft, loBottom,hiBottom
2710//      loLeft = V_min*0.98             //+/- 2%
2711//      hiLeft = V_max*1.02
2712       
2713//      SetAxis left loLeft,hiLeft
2714//      SetAxis bottom pnt2x(rescaledTime,pt-zoom),pnt2x(rescaledTime,pt+zoom)
2715       
2716        Cursor/P A rescaledTime pt              //at the point
2717        Cursor/P B rescaledTime pt              //at the same point
2718
2719        SetDataFolder root:
2720       
2721        return(0)
2722End
2723
2724Function EC_FindStepButton_down(ctrlName) : ButtonControl
2725        String ctrlName
2726       
2727//      Variable upDown = -5
2728        NVAR upDown = root:Packages:NIST:Event:gStepTolerance
2729       
2730        PutCursorsAtBigStep(upDown)
2731//      PutCursorsAtStep(-1*upDown)
2732
2733        return(0)
2734end
2735
2736
2737Function EC_FindStepButton_up(ctrlName) : ButtonControl
2738        String ctrlName
2739       
2740//      Variable upDown = 5
2741        NVAR upDown = root:Packages:NIST:Event:gStepTolerance
2742
2743        PutCursorsAtBigStep(upDown)
2744//      PutCursorsAtStep(upDown)
2745
2746        return(0)
2747end
2748
2749// if the Trim button section is uncommented, it's "Zap outlier"
2750//
2751Function EC_FindOutlierButton(ctrlName) : ButtonControl
2752        String ctrlName
2753       
2754        fFindOutlier()
2755//
2756//      STRUCT WMButtonAction ba
2757//      ba.eventCode = 2
2758//
2759//      EC_TrimPointsButtonProc(ba)
2760
2761        return(0)
2762end
2763
2764Function EC_DoDifferential(ctrlName) : ButtonControl
2765        String ctrlName
2766       
2767        DifferentiatedTime()
2768        DoWindow/F EventCorrectionPanel
2769       
2770        //if trace is not on graph, add it
2771        SetDataFolder root:Packages:NIST:Event:
2772
2773        String list = WaveList("*_DIF", ";", "WIN:EventCorrectionPanel")
2774        if(strlen(list) == 0)
2775                AppendToGraph/R rescaledTime_DIF
2776                ModifyGraph msize=1,rgb(rescaledTime_DIF)=(65535,0,0)
2777                ReorderTraces rescaledTime,{rescaledTime_DIF}           // put the differential behind the event data
2778        endif
2779        SetDataFolder root:
2780        return(0)
2781end
2782
2783//////////////   Custom Bins  /////////////////////
2784//
2785//
2786//
2787// make sure that the bins are defined and the waves exist before
2788// trying to draw the panel
2789//
2790Proc Show_CustomBinPanel()
2791        DoWindow/F CustomBinPanel
2792        if(V_flag ==0)
2793                Init_CustomBins()
2794                CustomBinPanel()
2795        EndIf
2796End
2797
2798
2799Function Init_CustomBins()
2800
2801        NVAR nSlice = root:Packages:NIST:Event:gEvent_nslices
2802        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2803
2804        Variable/G root:Packages:NIST:Event:gEvent_ForceTmaxBin=1               //==1 to enforce t_longest in user-defined custom bins
2805
2806        SetDataFolder root:Packages:NIST:Event:
2807               
2808        Make/O/D/N=(nSlice) timeWidth
2809        Make/O/D/N=(nSlice+1) binEndTime,binCount
2810       
2811        timeWidth = t_longest/nslice
2812        binEndTime = p
2813        binCount = p+1 
2814       
2815        SetDataFolder root:
2816       
2817        return(0)
2818End
2819
2820////////////////       
2821//
2822// Allow custom definitions of the bin widths
2823//
2824// Define by the number of bins, and the time width of each bin
2825//
2826// This shares the number of slices and the maximum time with the main panel
2827//
2828Proc CustomBinPanel()
2829        PauseUpdate; Silent 1           // building window...
2830        NewPanel /W=(130,44,851,455)/K=2 /N=CustomBinPanel
2831        DoWindow/C CustomBinPanel
2832        ModifyPanel fixedSize=1//,noEdit =1
2833        SetDrawLayer UserBack
2834       
2835        Button button0,pos={654,42}, size={50,20},title="Done",fSize=12
2836        Button button0,proc=CB_Done_Proc
2837        Button button1,pos={663,14},size={40,20},proc=CB_HelpButtonProc,title="?"
2838        Button button2,pos={216,42},size={80,20},title="Update",proc=CB_UpdateWavesButton       
2839        SetVariable setvar1,pos={23,13},size={160,20},title="Number of slices",fSize=12
2840        SetVariable setvar1,proc=CB_NumSlicesSetVarProc,value=root:Packages:NIST:Event:gEvent_nslices
2841        SetVariable setvar2,pos={24,44},size={160,20},title="Max Time (s)",fSize=10
2842        SetVariable setvar2,value=root:Packages:NIST:Event:gEvent_t_longest     
2843
2844        CheckBox chkbox1,pos={216,14},title="Enforce Max Time?"
2845        CheckBox chkbox1,variable = root:Packages:NIST:Event:gEvent_ForceTmaxBin
2846        Button button3,pos={500,14},size={90,20},proc=CB_SaveBinsButtonProc,title="Save Bins"
2847        Button button4,pos={500,42},size={100,20},proc=CB_ImportBinsButtonProc,title="Import Bins"     
2848               
2849        SetDataFolder root:Packages:NIST:Event:
2850
2851        Display/W=(291,86,706,395)/HOST=CustomBinPanel/N=BarGraph binCount vs binEndTime
2852        ModifyGraph mode=5
2853        ModifyGraph marker=19
2854        ModifyGraph lSize=2
2855        ModifyGraph rgb=(0,0,0)
2856        ModifyGraph msize=2
2857        ModifyGraph hbFill=2
2858        ModifyGraph gaps=0
2859        ModifyGraph usePlusRGB=1
2860        ModifyGraph toMode=1
2861        ModifyGraph useBarStrokeRGB=1
2862        ModifyGraph standoff=0
2863        SetAxis left 0,*
2864        Label bottom "\\Z14Time (seconds)"
2865        Label left "\\Z14Number of Events"
2866        SetActiveSubwindow ##
2867       
2868        // and the table
2869        Edit/W=(13,87,280,394)/HOST=CustomBinPanel/N=T0
2870        AppendToTable/W=CustomBinPanel#T0 timeWidth,binEndTime
2871        ModifyTable width(Point)=40
2872        SetActiveSubwindow ##
2873       
2874        SetDataFolder root:
2875       
2876EndMacro
2877
2878// save the bins - use Igor Text format
2879//
2880Function CB_SaveBinsButtonProc(ba) : ButtonControl
2881        STRUCT WMButtonAction &ba
2882
2883        switch( ba.eventCode )
2884                case 2: // mouse up
2885                        // click code here
2886
2887                        SetDataFolder root:Packages:NIST:Event:
2888
2889                        Wave timeWidth = timeWidth
2890                        Wave binEndTime = binEndTime
2891                       
2892                        Save/T timeWidth,binEndTime                     //will ask for a name
2893
2894                        break
2895                case -1: // control being killed
2896                        break
2897        endswitch
2898
2899        SetDataFolder root:
2900       
2901        return 0
2902End
2903
2904// Import the bins - use Igor Text format
2905//
2906// -- be sure that the number of bins is reset
2907// -?- how about the t_longest? - this should be set by the load, not here
2908//
2909// -- loads in timeWidth and binEndTime
2910//
2911Function CB_ImportBinsButtonProc(ba) : ButtonControl
2912        STRUCT WMButtonAction &ba
2913
2914        switch( ba.eventCode )
2915                case 2: // mouse up
2916                        // click code here
2917                        NVAR nSlice = root:Packages:NIST:Event:gEvent_nslices
2918
2919                        SetDataFolder root:Packages:NIST:Event:
2920
2921                        // prompt for the load of data
2922                        LoadWave/T/O
2923                        if(strlen(S_fileName) == 0)
2924                                //user cancelled
2925                                DoAlert 0,"No file selected, nothing done."
2926                                return(0)
2927                        endif
2928
2929                        Wave timeWidth = timeWidth
2930                        nSlice = numpnts(timeWidth)
2931                       
2932                        break
2933                case -1: // control being killed
2934                        break
2935        endswitch
2936
2937        SetDataFolder root:
2938       
2939        return 0
2940End
2941
2942
2943
2944//
2945// can either use the widths as stated -- then the end time may not
2946// match the actual end time of the data set
2947//
2948// -- or --
2949//
2950// enforce the end time of the data set to be the end time of the bins,
2951// then the last bin width must be reset to force the constraint
2952//
2953//
2954Function CB_UpdateWavesButton(ba) : ButtonControl
2955        STRUCT WMButtonAction &ba
2956
2957        switch( ba.eventCode )
2958                case 2: // mouse up
2959                        // click code here
2960                        NVAR nSlice = root:Packages:NIST:Event:gEvent_nslices
2961                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2962                        NVAR enforceTmax = root:Packages:NIST:Event:gEvent_ForceTmaxBin
2963                       
2964                        // update the waves, and recalculate everything for the display
2965                        SetDataFolder root:Packages:NIST:Event:
2966
2967                        Wave timeWidth = timeWidth
2968                        Wave binEndTime = binEndTime
2969                        Wave binCount = binCount
2970                       
2971                        // use the widths as entered
2972                        binEndTime[0] = 0
2973                        binEndTime[1,] = binEndTime[p-1] + timeWidth[p-1]
2974                       
2975                        // enforce the longest time as the end bin time
2976                        // note that this changes the last time width
2977                        if(enforceTmax)
2978                                binEndTime[nSlice] = t_longest
2979                                timeWidth[nSlice-1] = t_longest - binEndTime[nSlice-1]
2980                        endif
2981                       
2982                        binCount = p+1
2983                        binCount[nSlice] = 0            // last point is zero, just for display
2984//                      binCount *= sign(timeWidth)             //to alert to negative time bins
2985                       
2986                        // make the timeWidth bold and red if the widths are negative
2987                        WaveStats/Q timeWidth
2988                        if(V_min < 0)
2989                                ModifyTable/W=CustomBinPanel#T0 style(timeWidth)=1,rgb(timeWidth)=(65535,0,0)                   
2990                        else
2991                                ModifyTable/W=CustomBinPanel#T0 style(timeWidth)=0,rgb(timeWidth)=(0,0,0)                       
2992                        endif
2993                       
2994                        break
2995                case -1: // control being killed
2996                        break
2997        endswitch
2998
2999        SetDataFolder root:
3000       
3001        return 0
3002End
3003
3004Function CB_HelpButtonProc(ba) : ButtonControl
3005        STRUCT WMButtonAction &ba
3006
3007        switch( ba.eventCode )
3008                case 2: // mouse up
3009                        // click code here
3010                        DisplayHelpTopic/Z "Event Mode Data[Setting up Custom Bin Widths]"
3011                        break
3012                case -1: // control being killed
3013                        break
3014        endswitch
3015
3016        return 0
3017End
3018
3019Function CB_Done_Proc(ba) : ButtonControl
3020        STRUCT WMButtonAction &ba
3021       
3022        String win = ba.win
3023        switch (ba.eventCode)
3024                case 2:
3025                        DoWindow/K CustomBinPanel
3026                        break
3027        endswitch
3028        return(0)
3029End
3030
3031
3032Function CB_NumSlicesSetVarProc(sva) : SetVariableControl
3033        STRUCT WMSetVariableAction &sva
3034
3035        switch( sva.eventCode )
3036                case 1: // mouse up
3037                case 2: // Enter key
3038                case 3: // Live update
3039                        Variable dval = sva.dval
3040                        String sval = sva.sval
3041                        SetDataFolder root:Packages:NIST:Event:
3042
3043                        Wave timeWidth = timeWidth
3044                        Wave binEndTime = binEndTime
3045                       
3046                        Redimension/N=(dval) timeWidth
3047                        Redimension/N=(dval+1) binEndTime,binCount
3048                       
3049                        SetDataFolder root:
3050                       
3051                        break
3052                case -1: // control being killed
3053                        break
3054        endswitch
3055
3056        return 0
3057End
3058
3059
3060///////////////////
3061//
3062// utility to split a large file
3063// 100 MB is the recommended size
3064// events can be clipped here, so be sure to trim the ends of the
3065// resulting files as needed.
3066//
3067// - works like the unix 'split' command
3068//
3069//
3070
3071Proc SplitBigFile(splitSize, baseStr)
3072        Variable splitSize = 100
3073        String baseStr="split"
3074        Prompt splitSize,"Target file size, in MB (< 150 MB!!)"
3075        Prompt baseStr,"File prefix, number will be appended"
3076       
3077        if(splitSize > root:Packages:NIST:Event:gEventFileTooLarge)
3078                Abort "File split must be less than 150 MB. Please try again"
3079        endif
3080       
3081        fSplitBigFile(splitSize, baseStr)
3082       
3083        ShowSplitFileTable()
3084End
3085
3086Function/S fSplitBigFile(splitSize, baseStr)
3087        Variable splitSize
3088        String baseStr         
3089
3090
3091        String fileName=""              // File name, partial path, full path or "" for dialog.
3092        Variable refNum
3093        String str
3094        SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3095       
3096        listStr=""              //initialize output list
3097
3098        Variable readSize=1e6           //1 MB
3099        Make/O/B/U/N=(readSize) aBlob                   //1MB worth
3100        Variable numSplit
3101        Variable num,ii,jj,outRef,frac
3102        String thePath, outStr
3103       
3104        Printf "SplitSize = %u MB\r",splitSize
3105        splitSize = trunc(splitSize) * 1e6              // now in bytes
3106       
3107       
3108        // Open file for read.
3109        Open/R/Z=2/F="????"/P=catPathName refNum as fileName
3110        thePath = ParseFilePath(1, S_fileName, ":", 1, 0)
3111        Print "thePath = ",thePath
3112       
3113        // Store results from Open in a safe place.
3114        Variable err = V_flag
3115        String fullPath = S_fileName
3116
3117        if (err == -1)
3118                Print "cancelled by user."
3119                return ("")
3120        endif
3121
3122        FStatus refNum
3123       
3124        Printf "total # bytes = %u\r",V_logEOF
3125
3126        numSplit=0
3127        if(V_logEOF > splitSize)
3128                numSplit = trunc(V_logEOF/splitSize)
3129        endif
3130
3131        frac = V_logEOF - numSplit*splitSize
3132        Print "numSplit = ",numSplit
3133        Printf "frac = %u\r",frac
3134       
3135        num=0
3136        if(frac > readSize)
3137                num = trunc(frac/readSize)
3138        endif
3139
3140       
3141        frac = frac - num*readSize
3142
3143        Print "num = ",num
3144        Printf "frac = %u\r",frac
3145       
3146//      baseStr = "split"
3147       
3148        for(ii=0;ii<numSplit;ii+=1)
3149                outStr = (thePath+baseStr+num2str(ii)+".hst")
3150//              Print "outStr = ",outStr
3151                Open outRef as outStr
3152
3153                for(jj=0;jj<(splitSize/readSize);jj+=1)
3154                        FBinRead refNum,aBlob
3155                        FBinWrite outRef,aBlob
3156                endfor
3157
3158                Close outRef
3159//              listStr += outStr+";"           //no, this is the FULL path
3160                listStr += baseStr+num2str(ii)+".hst"+";"
3161        endfor
3162
3163        Make/O/B/U/N=(frac) leftover
3164        // ii was already incremented past the loop
3165        outStr = (thePath+baseStr+num2str(ii)+".hst")
3166        Open outRef as outStr
3167        for(jj=0;jj<num;jj+=1)
3168                FBinRead refNum,aBlob
3169                FBinWrite outRef,aBlob
3170        endfor
3171        FBinRead refNum,leftover
3172        FBinWrite outRef,leftover
3173
3174        Close outRef
3175//      listStr += outStr+";"
3176        listStr += baseStr+num2str(ii)+".hst"+";"
3177
3178        FSetPos refNum,V_logEOF
3179        Close refNum
3180       
3181        KillWaves/Z aBlob,leftover
3182        return(listStr)
3183End
3184
3185// allows the list of loaded files to be edited
3186Function ShowSplitFileTable()
3187
3188        SVAR str = root:Packages:NIST:Event:gSplitFileList
3189       
3190        WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3191        if(waveExists(tw) != 1)
3192                Make/O/T/N=1 root:Packages:NIST:Event:SplitFileWave
3193                WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3194        endif
3195
3196        List2TextWave(str,tw)
3197        Edit tw
3198
3199        return(0)
3200End
3201
3202
3203//// save the sliced data, and accumulate slices
3204//  *** this works with sliced data -- that is data that has been PROCESSED
3205//
3206// need some way of ensuring that the slices match up since I'm blindly adding them together.
3207//
3208// mode = 0             wipe out the old accumulated, copy slicedData to accumulatedData
3209// mode = 1             add current slicedData to accumulatedData
3210// mode = 2             copy accumulatedData to slicedData in preparation of export or display
3211// mode = 3             unused...
3212//
3213//      "Split Large File",SplitBigFile()
3214//      "Accumulate First Slice",AccumulateSlices(0)
3215//      "Add Current Slice",AccumulateSlices(1)
3216//      "Display Accumulated Slices",AccumulateSlices(2)       
3217//
3218Function AccumulateSlicesButton(ctrlName) : ButtonControl
3219        String ctrlName
3220       
3221        Variable mode
3222        mode = str2num(ctrlName[strlen(ctrlName)-1])
3223//      Print "mode=",mode
3224        AccumulateSlices(mode)
3225       
3226        return(0)
3227End
3228
3229// added an extra (optional) parameter for silent operation - to be used when a list of
3230// files are processed - so the user can walk away as it works. Using the buttons from the
3231// panel will still present the alerts
3232//
3233Function AccumulateSlices(mode,[skipAlert])
3234        Variable mode,skipAlert
3235       
3236        if(ParamIsDefault(skipAlert))
3237                skipAlert = 0
3238        endif
3239        SetDataFolder root:Packages:NIST:Event:
3240
3241        switch(mode)   
3242                case 0:
3243                        if(!skipAlert)
3244                                DoAlert 0,"The current data has been copied to the accumulated set. You are now ready to add more data."
3245                        endif
3246                        KillWaves/Z accumulatedData
3247                        Duplicate/O slicedData accumulatedData 
3248                        KillWaves/Z accum_BinCount              // the accumulated BinCount
3249                        Duplicate/O binCount accum_binCount     
3250                        break
3251                case 1:
3252                        if(!skipAlert)
3253                                DoAlert 0,"The current data has been added to the accumulated data. You can add more data."
3254                        endif
3255                        Wave acc=accumulatedData
3256                        Wave cur=slicedData
3257                        acc += cur
3258                        Wave binCount = binCount
3259                        Wave accum_binCount = accum_BinCount
3260                        accum_BinCount += binCount
3261                        binCount = accum_binCount                       // !! NOTE - the bin count wave is now the total
3262//                      AccumulateSlices(2,skipAlert=skipAlert)                                         // and a recursive call to show the total!
3263                        break
3264                case 2:
3265                        if(!skipAlert)
3266                                DoAlert 0,"The accumulated data is now the display data and is ready for display or export."
3267                        endif
3268                        Duplicate/O accumulatedData slicedData
3269                        // do something to "touch" the display to force it to update
3270                        NVAR gLog = root:Packages:NIST:Event:gEvent_logint
3271                        LogIntEvent_Proc("",gLog)
3272                        break
3273                default:                       
3274                               
3275        endswitch
3276
3277        SetDataFolder root:
3278        return(0)
3279end
3280
3281
3282////////////////////////////////////////////
3283//
3284// Panel and procedures for decimation
3285//
3286////////////////////////////////////////////
3287
3288//Function E_ShowDecimateButton(ctrlName) : ButtonControl
3289//      String ctrlName
3290//
3291//      DoWindow/F DecimatePanel
3292//      if(V_flag ==0)
3293//              Execute "DecimatePanel()"
3294//      endif
3295//End
3296//
3297//
3298//Proc DecimatePanel() //: Panel
3299//     
3300//      PauseUpdate; Silent 1           // building window...
3301//      NewPanel /W=(1602,44,1961,380)/K=1
3302////    ShowTools/A
3303//      Button button0,pos={29,15},size={100,20},proc=SplitFileButtonProc,title="Split Big File"
3304//      SetVariable setvar0,pos={182,55},size={150,15},title="Decimation factor",fsize=10
3305//      SetVariable setvar0,limits={1,inf,1},value= root:Packages:NIST:Event:gDecimation
3306//      Button button1,pos={26,245},size={150,20},proc=LoadDecimateButtonProc,title="Load and Decimate"
3307//      Button button2,pos={25,277},size={150,20},proc=ConcatenateButtonProc,title="Concatenate"
3308//      Button button3,pos={25,305},size={150,20},proc=DisplayConcatenatedButtonProc,title="Display Concatenated"
3309//      Button button4,pos={29,52},size={130,20},proc=Stream_LoadDecim,title="Load From List"
3310//     
3311//      GroupBox group0 title="Manual Controls",size={185,112},pos={14,220}
3312//EndMacro
3313
3314
3315Function SplitFileButtonProc(ctrlName) : ButtonControl
3316        String ctrlName
3317
3318        Execute "SplitBigFile()"
3319End
3320
3321
3322// show all of the data
3323//
3324Proc ShowDecimatedGraph()
3325
3326        DoWindow/F DecimatedGraph
3327        if(V_flag == 0)
3328                PauseUpdate; Silent 1           // building window...
3329                String fldrSav0= GetDataFolder(1)
3330                SetDataFolder root:Packages:NIST:Event:
3331                Display /W=(25,44,486,356)/K=1/N=DecimatedGraph rescaledTime_dec
3332                SetDataFolder fldrSav0
3333                ModifyGraph mode=4
3334                ModifyGraph marker=19
3335                ModifyGraph rgb(rescaledTime_dec)=(0,0,0)
3336                ModifyGraph msize=1
3337                ErrorBars rescaledTime_dec OFF
3338                Label left "\\Z14Time (seconds)"
3339                Label bottom "\\Z14Event number"
3340                ShowInfo
3341        endif
3342       
3343EndMacro
3344
3345// data has NOT been processed
3346//
3347// so work with x,y,t, and rescaled time
3348// variables -- t_longest
3349Function ConcatenateButtonProc(ctrlName) : ButtonControl
3350        String ctrlName
3351       
3352        DoAlert 1,"Is this the first file?"
3353        Variable first = V_flag
3354       
3355        fConcatenateButton(first)
3356       
3357        return(0)
3358End
3359
3360Function fConcatenateButton(first)
3361        Variable first
3362
3363
3364        SetDataFolder root:Packages:NIST:Event:
3365
3366        Wave timePt_dTmp=timePt_dTmp
3367        Wave xLoc_dTmp=xLoc_dTmp
3368        Wave yLoc_dTmp=yLoc_dTmp
3369        Wave rescaledTime_dTmp=rescaledTime_dTmp
3370       
3371        NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3372        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3373       
3374       
3375        if(first==1)            //1==yes, 2==no
3376                //then copy the files over, adjusting the time to start from zero
3377                // rescaledTime starts from zero (set by the loader)
3378
3379                timePt_dTmp -= timePt_dTmp[0]                   //subtract the first value
3380               
3381                Duplicate/O timePt_dTmp timePt_dec
3382                Duplicate/O xLoc_dTmp xLoc_dec
3383                Duplicate/O yLoc_dTmp yLoc_dec
3384                Duplicate/O rescaledTime_dTmp rescaledTime_dec
3385               
3386                t_longest_dec = t_longest
3387       
3388        else
3389                // concatenate the files + adjust the time
3390                Wave timePt_dec=timePt_dec
3391                Wave xLoc_dec=xLoc_dec
3392                Wave yLoc_dec=yLoc_dec
3393                Wave rescaledTime_dec=rescaledTime_dec
3394
3395                // adjust the times -- assuming they add
3396                // rescaledTime starts from zero (set by the loader)
3397                //
3398                //
3399                rescaledTime_dTmp += rescaledTime_dec[numpnts(rescaledTime_dec)-1]
3400                rescaledTime_dTmp += abs(rescaledTime_dec[numpnts(rescaledTime_dec)-1] - rescaledTime_dec[numpnts(rescaledTime_dec)-2])
3401               
3402                timePt_dTmp -= timePt_dTmp[0]                   //subtract the first value     
3403               
3404                timePt_dTmp += timePt_dec[numpnts(timePt_dec)-1]                // offset by the last point
3405                timePt_dTmp += abs(timePt_dec[numpnts(timePt_dec)-1] - timePt_dec[numpnts(timePt_dec)-2])               // plus delta so there's not a flat step
3406               
3407                Concatenate/NP/O {timePt_dec,timePt_dTmp}, tmp
3408                Duplicate/O tmp timePt_dec
3409               
3410                Concatenate/NP/O {xLoc_dec,xLoc_dTmp}, tmp
3411                Duplicate/O tmp xLoc_dec
3412               
3413                Concatenate/NP/O {yLoc_dec,yLoc_dTmp}, tmp
3414                Duplicate/O tmp yLoc_dec
3415               
3416                Concatenate/NP/O {rescaledTime_dec,rescaledTime_dTmp}, tmp
3417                Duplicate/O tmp rescaledTime_dec
3418               
3419
3420                KillWaves tmp
3421
3422                t_longest_dec = rescaledTime_dec[numpnts(rescaledTime_dec)-1]
3423
3424        endif
3425       
3426       
3427        SetDataFolder root:
3428       
3429        return(0)
3430
3431End
3432
3433Function DisplayConcatenatedButtonProc(ctrlName) : ButtonControl
3434        String ctrlName
3435
3436        //copy the files over to the display set for processing
3437        SetDataFolder root:Packages:NIST:Event:
3438
3439        Wave timePt_dec=timePt_dec
3440        Wave xLoc_dec=xLoc_dec
3441        Wave yLoc_dec=yLoc_dec
3442        Wave rescaledTime_dec=rescaledTime_dec
3443               
3444        NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3445        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3446       
3447        Duplicate/O timePt_dec timePt
3448        Duplicate/O xLoc_dec xLoc
3449        Duplicate/O yLoc_dec yLoc
3450        Duplicate/O rescaledTime_dec rescaledTime
3451       
3452        t_longest = t_longest_dec       
3453       
3454        SetDataFolder root:
3455       
3456        return(0)
3457
3458End
3459
3460
3461Proc DisplayForSlicing()
3462
3463        // plot the EventBarGraph?
3464                SetDataFolder root:Packages:NIST:Event:
3465                Display /W=(110,705,610,1132)/N=SliceGraph /K=1 binCount vs binEndTime
3466                ModifyGraph mode=5
3467                ModifyGraph marker=19
3468                ModifyGraph lSize=2
3469                ModifyGraph rgb=(0,0,0)
3470                ModifyGraph msize=2
3471                ModifyGraph hbFill=2
3472                ModifyGraph gaps=0
3473                ModifyGraph usePlusRGB=1
3474                ModifyGraph toMode=1
3475                ModifyGraph useBarStrokeRGB=1
3476                ModifyGraph standoff=0
3477
3478                SetDataFolder root:
3479
3480// append the differential
3481        AppendToGraph/R root:Packages:NIST:Event:rescaledTime_DIF vs root:Packages:NIST:Event:rescaledTime
3482        ModifyGraph rgb(rescaledTime_DIF)=(1,16019,65535)
3483
3484End
3485
3486
3487
3488// unused, old testing procedure
3489Function LoadDecimateButtonProc(ctrlName) : ButtonControl
3490        String ctrlName
3491
3492        LoadEventLog_Button("")
3493       
3494        // now decimate
3495        SetDataFolder root:Packages:NIST:Event:
3496
3497        Wave timePt=timePt
3498        Wave xLoc=xLoc
3499        Wave yLoc=yLoc
3500        NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3501
3502        NVAR decimation = root:Packages:NIST:Event:gDecimation
3503
3504
3505        Duplicate/O timePt, timePt_dTmp
3506        Duplicate/O xLoc, xLoc_dTmp
3507        Duplicate/O yLoc, yLoc_dTmp
3508        Resample/DOWN=(decimation)/N=1 timePt_dTmp
3509        Resample/DOWN=(decimation)/N=1 xLoc_dTmp
3510        Resample/DOWN=(decimation)/N=1 yLoc_dTmp
3511
3512
3513        Duplicate/O timePt_dTmp rescaledTime_dTmp
3514        rescaledTime_dTmp = 1e-7*(timePt_dTmp - timePt_dTmp[0])         //convert to seconds and start from zero
3515        t_longest_dec = waveMax(rescaledTime_dTmp)              //should be the last point
3516
3517        SetDataFolder root:
3518
3519       
3520End
3521
3522
3523
3524
3525
3526
3527
3528//
3529// loads a list of files, decimating each chunk as it is read in
3530//
3531Function Stream_LoadDecim(ctrlName)
3532        String ctrlName
3533       
3534        Variable fileref
3535        Variable num,ii
3536       
3537        SVAR filename = root:Packages:NIST:Event:gEvent_logfile
3538        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3539
3540        SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3541        NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3542        NVAR decimation = root:Packages:NIST:Event:gDecimation
3543
3544        String pathStr
3545        PathInfo catPathName
3546        pathStr = S_Path
3547
3548// if "stream" mode is not checked - abort
3549        NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
3550        if(gEventModeRadioVal != MODE_STREAM)
3551                Abort "The mode must be 'Stream' to use this function"
3552                return(0)
3553        endif
3554
3555// if the list has been edited, turn it into a list
3556        WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3557        if(WaveExists(tw))
3558                listStr = TextWave2SemiList(tw)
3559        else
3560                ShowSplitFileTable()
3561                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3562                return(0)
3563        endif
3564       
3565
3566        //loop through everything in the list
3567        num = ItemsInList(listStr)
3568        if(num == 0)
3569                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3570                return (0)
3571        endif
3572       
3573        for(ii=0;ii<num;ii+=1)
3574
3575// (1) load the file, prepending the path               
3576                filename = pathStr + StringFromList(ii, listStr  ,";")
3577                if(EventDataType(filename) != 1)                //if not hst extension of raw data
3578                        Abort "Data must be raw event data with file extension .hst for this operation"
3579                        return(0)
3580                endif           
3581
3582#if (exists("EventLoadWave")==4)
3583                LoadEvents_XOP()
3584#else
3585                LoadEvents()
3586#endif 
3587
3588                SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root:
3589
3590                Wave timePt=timePt
3591                Wave xLoc=xLoc
3592                Wave yLoc=yLoc
3593                CleanupTimes(xLoc,yLoc,timePt)          //remove zeroes
3594
3595                Duplicate/O timePt rescaledTime
3596                rescaledTime = 1e-7*(timePt-timePt[0])          //convert to seconds and start from zero
3597                t_longest = waveMax(rescaledTime)               //should be the last point
3598               
3599// (2) do the decimation, just on timePt. create rescaledTime from the decimated timePt
3600               
3601                Duplicate/O timePt, timePt_dTmp
3602                Duplicate/O xLoc, xLoc_dTmp
3603                Duplicate/O yLoc, yLoc_dTmp
3604                Resample/DOWN=(decimation)/N=1 timePt_dTmp
3605                Resample/DOWN=(decimation)/N=1 xLoc_dTmp
3606                Resample/DOWN=(decimation)/N=1 yLoc_dTmp
3607       
3608       
3609                Duplicate/O timePt_dTmp rescaledTime_dTmp
3610                rescaledTime_dTmp = 1e-7*(timePt_dTmp - timePt_dTmp[0])         //convert to seconds and start from zero
3611                t_longest_dec = waveMax(rescaledTime_dTmp)              //should be the last point
3612               
3613
3614// (3) concatenate the files + adjust the time
3615                fConcatenateButton(ii+1)                //passes 1 for the first time, >1 each other time
3616       
3617        endfor
3618
3619////            Now that everything is decimated and concatenated, create the rescaled time wave
3620//      SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root:
3621//      Wave timePt_dec = timePt_dec
3622//      Duplicate/O timePt_dec rescaledTime_dec
3623//      rescaledTime_dec = 1e-7*(timePt_dec - timePt_dec[0])            //convert to seconds and start from zero
3624//      t_longest_dec = waveMax(rescaledTime_dec)               //should be the last point
3625       
3626        DisplayConcatenatedButtonProc("")
3627       
3628        SetDataFolder root:
3629
3630        return(0)
3631End
3632
3633Function ShowList_ToLoad(ctrlName)
3634        String ctrlName
3635       
3636        ShowSplitFileTable()
3637       
3638        return(0)
3639End
3640
3641
3642//
3643// loads a list of files that have been adjusted and saved
3644// -- does not decimate
3645//
3646Function Stream_LoadAdjustedList(ctrlName)
3647        String ctrlName
3648       
3649        Variable fileref
3650
3651        SVAR filename = root:Packages:NIST:Event:gEvent_logfile
3652        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3653
3654        SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3655        NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3656//      NVAR decimation = root:Packages:NIST:Event:gDecimation
3657
3658        String pathStr
3659        PathInfo catPathName
3660        pathStr = S_Path
3661
3662// if "stream" mode is not checked - abort
3663        NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
3664        if(gEventModeRadioVal != MODE_STREAM)
3665                Abort "The mode must be 'Stream' to use this function"
3666                return(0)
3667        endif
3668
3669// if the list has been edited, turn it into a list
3670        WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3671        if(WaveExists(tw))
3672                listStr = TextWave2SemiList(tw)
3673        else
3674                ShowSplitFileTable()
3675                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3676                return(0)
3677        endif
3678       
3679
3680        //loop through everything in the list
3681        Variable num,ii
3682        num = ItemsInList(listStr)
3683       
3684        if(num == 0)
3685                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3686                return(0)
3687        endif
3688       
3689        for(ii=0;ii<num;ii+=1)
3690
3691// (1) load the file, prepending the path               
3692                filename = pathStr + StringFromList(ii, listStr  ,";")
3693                if(EventDataType(filename) != 2)                //if not itx extension of edited data
3694                        Abort "Data must be edited data with file extension .itx for this operation"
3695                        return(0)
3696                endif           
3697                SetDataFolder root:Packages:NIST:Event:
3698                LoadWave/T/O fileName
3699
3700                SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root: ??
3701
3702// this is what is loaded -- _dec extension is what is concatenated, and will be copied back later
3703                Wave timePt=timePt
3704                Wave xLoc=xLoc
3705                Wave yLoc=yLoc
3706                Wave rescaledTime=rescaledTime
3707
3708//              CleanupTimes(xLoc,yLoc,timePt)          //remove zeroes
3709
3710//              Duplicate/O timePt rescaledTime
3711//              rescaledTime = 1e-7*(timePt-timePt[0])          //convert to seconds and start from zero
3712//              t_longest = waveMax(rescaledTime)               //should be the last point
3713               
3714// (2) No decimation
3715               
3716                Duplicate/O timePt, timePt_dTmp
3717                Duplicate/O xLoc, xLoc_dTmp
3718                Duplicate/O yLoc, yLoc_dTmp
3719                Duplicate/O rescaledTime, rescaledTime_dTmp
3720
3721
3722// (3) concatenate the files + adjust the time
3723                fConcatenateButton(ii+1)                //passes 1 for the first time, >1 each other time
3724       
3725        endfor
3726       
3727        DisplayConcatenatedButtonProc("")               // this resets the longest time, too
3728               
3729        SetDataFolder root:
3730
3731        return(0)
3732End
3733
3734//
3735// loads a list of files that have been adjusted and saved
3736// -- does not decimate
3737// -- bins as they are loaded
3738//
3739Function Stream_LoadAdjList_BinOnFly(ctrlName)
3740        String ctrlName
3741       
3742        Variable fileref
3743
3744        SVAR filename = root:Packages:NIST:Event:gEvent_logfile
3745        SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3746
3747        String pathStr
3748        PathInfo catPathName
3749        pathStr = S_Path
3750
3751// if "stream" mode is not checked - abort
3752        NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
3753        if(gEventModeRadioVal != MODE_STREAM)
3754                Abort "The mode must be 'Stream' to use this function"
3755                return(0)
3756        endif
3757
3758// if the list has been edited, turn it into a list
3759        WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3760        if(WaveExists(tw))
3761                listStr = TextWave2SemiList(tw)
3762        else
3763                ShowSplitFileTable()
3764                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3765                return(0)
3766        endif
3767       
3768
3769        //loop through everything in the list
3770        Variable num,ii,jj
3771        num = ItemsInList(listStr)
3772        if(num == 0)
3773                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3774                return(0)
3775        endif
3776       
3777////////////////
3778// declarations for the binning
3779// for the first pass, do all of this
3780        Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
3781       
3782        Wave binnedData = root:Packages:NIST:Event:binnedData
3783        Wave xLoc = root:Packages:NIST:Event:xLoc
3784        Wave yLoc = root:Packages:NIST:Event:yLoc
3785
3786// now with the number of slices and max time, process the events
3787
3788        NVAR yesSortStream = root:Packages:NIST:Event:gSortStreamEvents         //do I sort the events?
3789        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3790        NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
3791//      NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3792        NVAR t_segment_start = root:Packages:NIST:Event:gEvent_t_segment_start
3793        NVAR t_segment_end = root:Packages:NIST:Event:gEvent_t_segment_end
3794               
3795        SetDataFolder root:Packages:NIST:Event          //don't count on the folder remaining here
3796       
3797        Make/D/O/N=(128,128,nslices) slicedData
3798       
3799        Wave slicedData = slicedData
3800        Wave rescaledTime = rescaledTime
3801        Make/O/D/N=(128,128) tmpData
3802        Make/O/D/N=(nslices+1) binEndTime,binCount//,binStartTime
3803        Make/O/D/N=(nslices) timeWidth
3804        Wave binEndTime = binEndTime
3805        Wave timeWidth = timeWidth
3806        Wave binCount = binCount
3807
3808        slicedData = 0
3809        binCount = 0
3810
3811        variable p1,p2,t1,t2
3812
3813        t_segment_start = 0
3814// start w/file 1??     
3815        for(jj=0;jj<num;jj+=1)
3816
3817// (1) load the file, prepending the path               
3818                filename = pathStr + StringFromList(jj, listStr  ,";")
3819                if(EventDataType(filename) != 2)                //if not itx extension of edited data
3820                        Abort "Data must be edited data with file extension .itx for this operation"
3821                        return(0)
3822                endif           
3823                SetDataFolder root:Packages:NIST:Event:
3824                LoadWave/T/O fileName
3825
3826                SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root: ??
3827
3828// this is what is loaded --
3829                Wave timePt=timePt
3830                Wave xLoc=xLoc
3831                Wave yLoc=yLoc
3832                Wave rescaledTime=rescaledTime
3833
3834                if(jj==0)
3835                        timePt -= timePt[0]                     //make sure the first point is zero
3836                        t_segment_start = 0
3837                        t_segment_end = rescaledTime[numpnts(rescaledTime)-1]
3838                else
3839                        t_segment_start = t_segment_end
3840                        t_segment_end += rescaledTime[numpnts(rescaledTime)-1]
3841                       
3842                        rescaledTime += t_segment_start         
3843                endif
3844               
3845                Printf "jj=%g\t\tt_segment_start=%g\tt_segment_end=%g\r",jj,t_segment_start,t_segment_end
3846               
3847// this is the binning--
3848
3849//              del = t_longest/nslices
3850       
3851        // TODO
3852        // the global exists for this switch, but it is not implemented - not sure whether
3853        // it's correct to implement this at all --
3854        //
3855                if(yesSortStream == 1)
3856                        SortTimeData()
3857                endif
3858               
3859        // index the events before binning
3860        // if there is a sort of these events, I need to re-index the events for the histogram
3861        //      SetDataFolder root:Packages:NIST:Event
3862                IndexForHistogram(xLoc,yLoc,binnedData)
3863        //      SetDataFolder root:
3864                Wave index = root:Packages:NIST:Event:SavedIndex                //the index for the histogram
3865               
3866                for(ii=0;ii<nslices;ii+=1)
3867                        if(ii==0)
3868        //                      t1 = ii*del
3869        //                      t2 = (ii+1)*del
3870                                p1 = BinarySearch(rescaledTime,0)
3871                                p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
3872                        else
3873        //                      t2 = (ii+1)*del
3874                                p1 = p2+1               //one more than the old one
3875                                p2 = BinarySearch(rescaledTime,binEndTime[ii+1])               
3876                        endif
3877       
3878                        if(p1 == -1)
3879                                Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
3880                                p1 = 0          //set to the first point if it's off the end
3881                        Endif
3882                        if(p2 == -2)
3883                                Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
3884                                p2 = numpnts(rescaledTime)-1            //set to the last point if it's off the end
3885                        Endif
3886        //              Print p1,p2
3887       
3888                        tmpData=0
3889                        JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
3890        //              slicedData[][][ii] = tmpData[p][q]
3891                        slicedData[][][ii] += tmpData[p][q]
3892                       
3893//                      Duplicate/O tmpData,$("tmpData_"+num2str(ii)+"_"+num2str(jj))
3894                       
3895        //              binEndTime[ii+1] = t2
3896        //              binCount[ii] = sum(tmpData,-inf,inf)
3897                        binCount[ii] += sum(tmpData,-inf,inf)
3898                endfor
3899       
3900        endfor  //end loop over the file list
3901
3902// after all of the files have been loaded
3903        t_longest = t_segment_end               //this is the total time
3904
3905        Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
3906        Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
3907        logslicedData = log(slicedData)
3908       
3909               
3910        SetDataFolder root:
3911
3912        return(0)
3913End
3914
3915
3916/////////////////////////////////////
3917
3918// dd-mon-yyyy hh:mm:ss -> Event file name
3919// the VAX uses 24 hr time for hh
3920//
3921// scans as string elements since I'm reconstructing a string name
3922Function/S DateAndTime2HSTName(dateandtime)
3923        string dateAndTime
3924       
3925        String day,yr,hh,mm,ss,time_secs
3926        Variable mon
3927        string str,monStr,fileStr
3928       
3929        str=dateandtime
3930        sscanf str,"%2s-%3s-%4s %2s:%2s:%2s",day,monStr,yr,hh,mm,ss
3931        mon = monStr2num(monStr)
3932
3933        fileStr = "Event"+yr+num2str(mon)+day+hh+mm+ss+".hst"
3934        Print fileStr
3935
3936        return(fileStr)
3937end
3938
3939// dd-mon-yyyy hh:mm:ss -> Event file name
3940// the VAX uses 24 hr time for hh
3941//
3942// scans as string elements since I'm reconstructing a string name
3943Function DateAndTime2HSTNumber(dateandtime)
3944        string dateAndTime
3945       
3946        String day,yr,hh,mm,ss,time_secs
3947        Variable mon,num
3948        string str,monStr,fileStr
3949       
3950        str=dateandtime
3951        sscanf str,"%2s-%3s-%4s %2s:%2s:%2s",day,monStr,yr,hh,mm,ss
3952        mon = monStr2num(monStr)
3953
3954        fileStr = yr+num2str(mon)+day+hh+mm+ss
3955        num = str2num(fileStr)
3956
3957        return(num)
3958end
3959
3960Function HSTName2Num(str)
3961        String str
3962       
3963        Variable num
3964        sscanf str,"Event%d.hst",num
3965        return(num)
3966end
3967/////////////////////////////
3968
3969Proc GetListofITXorSplitFiles(searchStr)
3970        String searchStr = ".itx"
3971        ShowList_ToLoad("")
3972        GetListofITXHST(searchStr)
3973end
3974
3975// needs SANS_Utilites.ipf
3976Function GetListofITXHST(searchStr)
3977        String searchStr
3978       
3979        String fullList = IndexedFile(catpathName, -1, "????" )
3980       
3981        // list is currently everything -- trim it
3982        searchStr = "*" + searchStr + "*"
3983        String list = ListMatch(fullList, searchStr )
3984       
3985        Print "searchString = ",searchStr
3986        //
3987        Wave/T tw = root:Packages:NIST:Event:SplitFileWave
3988        List2TextWave(list,tw)
3989        Sort/A tw,tw                    //alphanumeric sort
3990        Edit tw
3991        return(0)
3992End
3993
3994//
3995// loads a list of files that have been adjusted and saved
3996// -- MUST run one through first manually to set the bin sapcing, then this
3997//    won't touch that, just process all of the files
3998// -- bins as they are loaded
3999//
4000Function Osc_LoadAdjList_BinOnFly(ctrlName)
4001        String ctrlName
4002       
4003        Variable fileref
4004
4005        SVAR filename = root:Packages:NIST:Event:gEvent_logfile
4006        SVAR listStr = root:Packages:NIST:Event:gSplitFileList
4007
4008        String pathStr,fileStr,tmpStr
4009        PathInfo catPathName
4010        pathStr = S_Path
4011
4012// if "Osc" mode is not checked - abort
4013        NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
4014        if(gEventModeRadioVal != MODE_OSCILL)
4015                Abort "The mode must be 'Oscillatory' to use this function"
4016                return(0)
4017        endif
4018
4019// if the list has been edited, turn it into a list
4020        WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
4021        if(WaveExists(tw))
4022                listStr = TextWave2SemiList(tw)
4023        else
4024                ShowSplitFileTable()
4025                DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
4026                return(0)
4027        endif
4028       
4029
4030        //loop through everything in the list
4031        Variable num,ii,jj
4032        num = ItemsInList(listStr)
4033
4034//handle the first file differently
4035//(1) load (import edited)
4036        jj=0
4037        filename = pathStr + StringFromList(jj, listStr  ,";")
4038        if(EventDataType(filename) != 2)                //if not itx extension of edited data
4039                Abort "Data must be edited data with file extension .itx for this operation"
4040                return(0)
4041        endif
4042       
4043        SetDataFolder root:Packages:NIST:Event:
4044        LoadWave/T/O fileName
4045        SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root: ??
4046
4047        // clear out the old sort index, if present, since new data is being loaded
4048        KillWaves/Z OscSortIndex       
4049        SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
4050        sprintf tmpStr, "%s: a user-modified event file\r",StringFromList(jj, listStr  ,";")
4051        dispStr = tmpStr       
4052       
4053//(2) bin
4054        Osc_ProcessEventLog("")
4055       
4056//(3) display bin details
4057        Execute "ShowBinTable()"
4058        Execute "BinEventBarGraph()"
4059       
4060//(4) add first
4061        AccumulateSlices(0,skipAlert=1)         //bypass the dialog
4062
4063        // with remaining files
4064        // load
4065        // bin
4066        // add next
4067
4068// start w/file 1, since "0th" file was already processed
4069        for(jj=1;jj<num;jj+=1)
4070       
4071        //(1) load (import edited)
4072                filename = pathStr + StringFromList(jj, listStr  ,";")
4073                if(EventDataType(filename) != 2)                //if not itx extension of edited data
4074                        Abort "Data must be edited data with file extension .itx for this operation"
4075                        return(0)
4076                endif
4077                SetDataFolder root:Packages:NIST:Event:
4078                LoadWave/T/O fileName
4079                SetDataFolder root:Packages:NIST:Event:                 //LoadEvents sets back to root: ??
4080       
4081                // clear out the old sort index, if present, since new data is being loaded
4082                KillWaves/Z OscSortIndex       
4083                SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
4084                sprintf tmpStr, "%s: a user-modified event file\r",StringFromList(jj, listStr  ,";")
4085                dispStr = tmpStr       
4086               
4087        //(2) bin
4088                Osc_ProcessEventLog("")
4089               
4090        //(3) display bin details
4091                Execute "ShowBinTable()"
4092                Execute "BinEventBarGraph()"
4093               
4094        //(4) add next
4095                AccumulateSlices(1,skipAlert=1)
4096       
4097        endfor  //end loop over the file list
4098
4099       
4100        //display total
4101        AccumulateSlices(2,skipAlert=1)
4102
4103        // let user know we're done
4104        DoAlert 0,"The list has been processed and is ready for export"
4105        SetDataFolder root:
4106
4107        return(0)
4108End
4109
4110
4111//returns 0 for unknown data
4112// 1 for raw hst files
4113// 2 for edited ITX files
4114//
4115//  TODO
4116//  -- currently, this blindly assumes an .hst extension for raw HST data, and .itx for edited data
4117//
4118Function EventDataType(filename)
4119        string filename
4120       
4121        Variable type
4122        Variable offset = strsearch(filename,".",Inf,1)         //work backwards
4123        String extension = filename[offset+1,strlen(filename)-1]
4124        //Print extension
4125        type = 0
4126        if(cmpstr(extension,"hst") == 0)
4127                type = 1
4128        endif
4129        if(cmpstr(extension,"itx") == 0)
4130                type = 2
4131        endif
4132        return(type)
4133end
4134
4135////////////
4136// this could be jazzed up quite a bit, but it is a first pass at
4137// figuring out what is going on with the events
4138//
4139Proc DecodeBinaryEvents(numToPrint)
4140        Variable numToPrint=10
4141       
4142        DecodeEvents(numToPrint)
4143end
4144
4145//      sscanf %x takes hex input and gives a real value output
4146// printf %d val gives the integer representation of the sscanf output
4147// printf %b val gives the binary representation.
4148Function DecodeEvents(numToPrint)
4149        Variable numToPrint
4150
4151        String filePathStr,buffer
4152        Variable fileRef,type,dataval,num0,num1,num2,num3
4153        Variable numT0,numPP,numZero,numXYevents,bit29
4154        Variable ii
4155
4156        ii=0
4157        Open/R fileref
4158        tic()
4159        do
4160                do
4161                        FReadLine fileref, buffer                       //skip the "blank" lines that have one character
4162                while(strlen(buffer) == 1)             
4163
4164                if (strlen(buffer) == 0)
4165                        break
4166                endif
4167               
4168                sscanf buffer,"%x",dataval
4169                if(ii<numToPrint)
4170                        printf "%s  %0.32b  ",buffer[0,7],dataval
4171                endif
4172               
4173                // two most sig bits (31-30)
4174                type = (dataval & 0xC0000000)/1073741824                //right shift by 2^30
4175
4176                if(ii<numToPrint)
4177                        printf "%0.2b  ",type
4178                endif
4179                                               
4180                if(type == 0)
4181                        num0 += 1
4182                        numXYevents += 1
4183                endif
4184                if(type == 2)
4185                        num2 += 1
4186                        numXYevents += 1
4187                endif
4188                if(type == 1)
4189                        num1 += 1
4190                endif
4191                if(type == 3)
4192                        num3 += 1
4193                endif   
4194               
4195                bit29 = (dataval & 0x20000000)/536870912                //bit 29 only , shift by 2^29
4196                if(ii<numToPrint)
4197                        printf "%0.1b  ",bit29
4198                endif
4199                               
4200                if(type==0 || type==2)
4201                        numPP += round(bit29)
4202                endif
4203               
4204                if(type==1 || type==3)
4205                        numT0 += round(bit29)
4206                endif
4207               
4208                if(dataval == 0)
4209                        numZero += 1
4210                endif
4211               
4212                if(ii<numToPrint)
4213                        printf "\r"
4214                        ii += 1
4215                endif
4216               
4217        while(1)
4218        Close fileref
4219toc()
4220
4221        Print "(Igor) numT0 = ",numT0   
4222        Print "num0 = ",num0   
4223        Print "num1 = ",num1   
4224        Print "num2 = ",num2   
4225        Print "num3 = rollover = ",num3
4226        Print "numXY = ",numXYevents
4227        Print "numZero = ",numZero
4228        Print "numPP = ",numPP
4229       
4230        return(0)
4231
4232End
4233
4234
4235//////////////////
4236Function FixStepDown()
4237       
4238        SetDataFolder root:Packages:NIST:Event:
4239       
4240        Wave rescaledTime = rescaledTime
4241        Wave timePt = timePt
4242        Variable rollTime,rollTicks,ii,delta
4243       
4244        rollTicks = 2^26                                // in ticks
4245        rollTime = 2^26*1e-7            // in seconds
4246
4247        for(ii=0;ii<numpnts(rescaledTime)-1;ii+=1)
4248                delta = rescaledTime[ii+1] - rescaledTime[ii]
4249                if(delta < -6.6 && delta > -6.8)                //assume anything this large is a step down
4250                print ii, delta
4251                        MultiThread timePt[ii+1,] += rollTicks
4252                        MultiThread rescaledTime[ii+1,] += rollTime
4253               
4254                        // updates the longest time (as does every operation of adjusting the data)
4255                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
4256                        t_longest = waveMax(rescaledTime)
4257                endif
4258       
4259        endfor
4260
4261        SetDataFolder root:
4262
4263        return(0)
4264End
4265
4266Function FixStepUp()
4267       
4268        SetDataFolder root:Packages:NIST:Event:
4269       
4270        Wave rescaledTime = rescaledTime
4271        Wave timePt = timePt
4272        Variable rollTime,rollTicks,ii,delta
4273       
4274        rollTicks = 2^26                                // in ticks
4275        rollTime = 2^26*1e-7            // in seconds
4276
4277        for(ii=0;ii<numpnts(rescaledTime)-1;ii+=1)
4278                delta = rescaledTime[ii+1] - rescaledTime[ii]
4279                if(delta > 6.6 && delta < 6.8)          //assume anything this large is a step up
4280                print ii, delta
4281                        MultiThread timePt[ii+1,] -= rollTicks
4282                        MultiThread rescaledTime[ii+1,] -= rollTime
4283               
4284                        // updates the longest time (as does every operation of adjusting the data)
4285                        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
4286                        t_longest = waveMax(rescaledTime)
4287                endif
4288       
4289        endfor
4290
4291        SetDataFolder root:
4292
4293        return(0)
4294End
4295
4296//    added a test function AutoFix_Rollover_Steps() that will search for steps (simple delta) that
4297//    are +/- 0.1s away from the 6.7s missed rollover. This function looks for up, down, then re-calculates
4298//    the derivative.
4299//  - this way, both the simple steps, and the "square" steps can both be corrected.
4300//
4301Function AutoFix_Rollover_Steps()
4302// fix steps up, then down (order doesn't matter)
4303        Print "Fixing steps of 6.7s up"
4304        FixStepUp()
4305        Print "Fixing steps of 6.7s down"
4306        FixStepDown()
4307//      re-do the differential
4308        DifferentiatedTime()
4309        return(0)
4310End
4311
4312
4313//upDown looks for a step up or down?
4314//
4315// plan is to be able to locate large (t0) steps from oscillations that are not timing errors
4316//
4317//
4318// will search from the leftmost cursor to the end. this allows skipping of oscillations
4319// that are not timing errors. It may introduce other issues, but we'll see what happens
4320//
4321Function PutCursorsAtBigStep(tol)
4322        Variable tol
4323       
4324        SetDataFolder root:Packages:NIST:Event:
4325
4326        Wave rescaledTime=rescaledTime
4327        Variable ii,delta,ptA,ptB,startPt,pt
4328               
4329        ptA = pcsr(A)
4330        ptB = pcsr(B)
4331        startPt = min(ptA,ptB)
4332               
4333        for(ii=startPt;ii<numpnts(rescaledTime)-1;ii+=1)
4334                delta = rescaledTime[ii+1] - rescaledTime[ii]           //if there is a step down, this will be negative
4335                if(abs(delta) > tol)
4336                        print ii, delta
4337                        pt = ii
4338                        break
4339                endif   
4340        endfor
4341       
4342        Cursor/P A rescaledTime pt+1    //at the point+1
4343        Cursor/P B rescaledTime numpnts(rescaledTime)-1         //at the end
4344
4345        SetDataFolder root:
4346
4347        return(0)
4348       
4349       
4350End
4351
4352
4353
4354Proc InsertTimeReset(period)
4355        Variable period
4356       
4357        fInsertTimeReset(period)
4358End
4359
4360
4361//
4362// for event data where a time reset signal was not sent, but the data is
4363// actually periodic. This function will reset the time at the input period.
4364// This assumes that you accurately know the period, and that the start time
4365// of the data coincides with the start time of the period, otherwise the
4366// time reset will happen in the middle of the cycle.
4367//
4368// period = period of reset (s)
4369//
4370Function fInsertTimeReset(period)
4371        Variable period
4372       
4373        SetDataFolder root:Packages:NIST:Event:
4374       
4375        Wave rescaledTime = rescaledTime
4376        Wave timePt = timePt
4377        Variable rollTime,rollTicks,ii,delta
4378       
4379        Variable period_ticks
4380       
4381        period_ticks = period*1e7               //period in ticks
4382       
4383
4384        for(ii=0;ii<numpnts(rescaledTime)-1;ii+=1)
4385                if(rescaledTime[ii] > period)
4386                        MultiThread timePt[ii,] -= period_ticks
4387                        MultiThread rescaledTime[ii,] -= period
4388                endif
4389        endfor
4390
4391// updates the longest time (as does every operation of adjusting the data)
4392        NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
4393        t_longest = waveMax(rescaledTime)
4394        SetDataFolder root:
4395
4396        return(0)
4397End
4398
Note: See TracBrowser for help on using the repository browser.