Context Navigation

EventModeProcessing.ipf @ 999

Last change on this file since 999 was 999, checked in by srkline, 7 years ago

changes to a few analysis models to make these Igor 7-ready

adding mask editing utilities

many changes to event mode for easier processing of split lists

updated event mode help file

+ lots more!

File size: 111.9 KB

Line
1	#pragma rtGlobals=1 // Use modern global access method.
2	#pragma IgorVersion=6.22
3
4	// vers 7.13e
5
6	// TODO:
7	//
8	// -- search for TODO for unresolved issues not on this list
9	//
10	// -- add comments to the code as needed
11	//
12	// -- write the help file, and link the help buttons to the help docs
13	//
14	// -- examples?
15	//
16	//
17	// -- NEW 2016
18	//
19	// -- see the Procedure DecodeBinaryEvents(numToPrint) for a first implementation of a "viewer"
20	// for the binary events. Prints out the decoded 32-bit binary to the command window for inspection
21	// Could be modified as needed to flag events, see what bad events really look like, etc. (May 2016)
22
23	// -- see the proc DisplayForSlicing() that may aid in setting the bins properly by plotting the
24	// bins along with the differential collection rate
25	//
26	// -- ?? need way to get correspondence between .hst files and VAX files? Names are all different. See
27	// DateAndTime2HSTName() functions and similar @ bottom of this file
28	//
29	// x- add the XOP to the distribution package
30	//
31	// x- Need to make sure that the rescaledTime and the differentiated time graphs are
32	// being properly updated when the data is processed, modified, etc.
33	//
34	// -- I need better nomenclature other than "stream" for the "continuous" data set.
35	// It's all a stream, just sometimes it's not oscillatory
36	//
37	//
38	// X- the slice display "fails" for data sets that have 3 or 4 slices, as the ModifyImage command
39	// interprets the data as being RGB - and so does nothing.
40	// need to find a way around this. This was fixed by displaying the data using the G=1 flag on AppendImage
41	// to prevent the "atuo-detection" of data as RGB
42	//
43	// -- Do something with the PP events. Currently, only the PP events that are XY (just the
44	// type 0 events (since I still need to find out what they realy mean)
45	//
46	// -- Add a switch to allow Sorting of the Stream data to remove the "time-reversed" data
47	// points. Maybe not kosher, but would clean things up.
48	//
49	// -- when a large stream file is split, and each segment is edited, it is re-loaded from a list and is currently
50	// NOT decimated, and there is no option. There could be an option for this, especially for these
51	// large files. The issue is that the data segments are concatentated (adjusting the time) and can
52	// be too large to handle (a 420 MB event file split and edited and re-loaded becomes a 1.6 GB Igor experiment!!)
53	//
54	// -- so allow the option of either decimation of the segments as they are loaded, or create the option to
55	// bin on the fly so that the full concatentated set does not need to be stored. Users could be
56	// directed to determine the binning and manipulate the data of a sufficiently decimated data set
57	// first to set the binning, then only process the full set.
58	//
59	///////////////////////////////
60	//
61	// NOTE -- to be able to show the T0 and PP event locations/times, force the loader to use the Igor code rather than
62	// the XOP. Then there will be waves generated with their locations: T0Time and T0EventNum, PPTime and PPEventNum. Edit these waves
63	// and (1) delete the zero points at the end of the waves and (2) multiply the Time wave * 1e-7 to convert to seconds.
64	// Then the waves can be plotted on top of the event data, so it can be seen where these events were identified.
65	//
66	/////////////// SWITCHES /////////////////
67	//
68	// for the "File Too Big" limit:
69	// Variable/G root:Packages:NIST:Event:gEventFileTooLarge = 150 // 150 MB considered too large
70	//
71	// for the tolerance of "step" detection
72	// Variable/G root:Packages:NIST:Event:gStepTolerance = 5 // 5 = # of standard deviations from mean. See PutCursorsAtStep()
73	//
74	//
75	///////// DONE //////////
76	//
77	// X- memory issues:
78	// -- in LoadEvents -- should I change the MAKE to:
79	// /I/U is unsigned 32-bit integer (for the time)
80	// /B/U is unsigned 8-bit integer (max val=255) for the x and y values
81	// -- then how does this affect downstream processing - such as rescaledTime, differentiation, etc.
82	// x- and can I re-write the XOP to create these types of data waves, and properly fill them...
83	//
84	// **- any integer waves must be translated by Igor into FP to be able to be displayed or for any
85	// type of analysis. so it's largely a waste of time to use integers. so simply force the XOP to
86	// generate only SP waves. this will at least save some space.
87	//
88	//
89	//
90	// x- Is there any way to improve the speed of the loader? How could an XOP be structured
91	// for maximum flexibility? Leave the post processing to Igor, but how much for the XOP
92	// to do? And can it handle such large amounts of data to pass back and forth, or
93	// does it need to be written as an operation, rather than a function??? I'd really
94	// rather that Igor handles the memory management, not me, if I write the XOP.
95	//
96	// **- as of 11/27, the OSX version of the XOP event loader is about 35x faster for the load!
97	// and is taking approx 1.8s/28MB, or about 6.5s/100MB of file. quite reasonable now, and
98	// probably a bit faster yet on the PC.
99	//
100	//
101	// X- fix the log/lin display - it's not working correctly
102	// I could use ModifyImage and log = 0\|1 keyword for the log Z display
103	// rather than creating a duplicate wave of log(data)
104	// -- it's in the Function sliceSelectEvent_Proc()
105	//
106	// X- add controls to show the bar graph
107	// x- add popup for selecting the binning type
108	// x- add ability to save the slices to RAW VAX files
109	// X- add control to show the bin counts and bin end times
110	// x- ADD buttons, switches, etc for the oscillatory mode - so that this can be accessed
111	//
112	// x- How are the headers filled for the VAX files from Teabag???
113	// -- I currently read the events 2x. Once to count the events to make the waves the proper
114	// size, then a second time to actualy process the events. Would it be faster to insert points
115	// as needed, or to estimate the size, and make it too large, then trim at the end...
116	// ((( NO -- I have no good way of getting a proper estimate of how many XY events there are for a file))
117	//
118	//
119	//
120
121
122	//
123	// These are also defined in the TISANE procedure file. In both files they are declared
124	// as Static, so they are local to each procedure
125	//
126	Static Constant ATXY = 0
127	Static Constant ATXYM = 2
128	Static Constant ATMIR = 1
129	Static Constant ATMAR = 3
130
131	Static Constant USECSPERTICK=0.1 // microseconds
132	Static Constant TICKSPERUSEC=10
133	Static Constant XBINS=128
134	Static Constant YBINS=128
135	//
136
137	Static Constant MODE_STREAM = 0
138	Static Constant MODE_OSCILL = 1
139	Static Constant MODE_TISANE = 2
140	Static Constant MODE_TOF = 3
141
142	//Menu "Macros"
143	// "Split Large File",SplitBigFile()
144	// "Accumulate First Slice",AccumulateSlices(0)
145	// "Add Current Slice",AccumulateSlices(1)
146	// "Display Accumulated Slices",AccumulateSlices(2)
147	//End
148
149
150
151	Proc Show_Event_Panel()
152	DoWindow/F EventModePanel
153	if(V_flag ==0)
154	Init_Event()
155	EventModePanel()
156	EndIf
157	End
158
159
160	Function Init_Event()
161
162	NewDataFolder/O/S root:Packages:NIST:Event
163
164	String/G root:Packages:NIST:Event:gEvent_logfile
165	String/G root:Packages:NIST:Event:gEventDisplayString="Details of the file load"
166
167	Variable/G root:Packages:NIST:Event:AIMTYPE_XY=0 // XY Event
168	Variable/G root:Packages:NIST:Event:AIMTYPE_XYM=2 // XY Minor event
169	Variable/G root:Packages:NIST:Event:AIMTYPE_MIR=1 // Minor rollover event
170	Variable/G root:Packages:NIST:Event:AIMTYPE_MAR=3 // Major rollover event
171
172	Variable/G root:Packages:NIST:Event:gEvent_time_msw = 0
173	Variable/G root:Packages:NIST:Event:gEvent_time_lsw = 0
174	Variable/G root:Packages:NIST:Event:gEvent_t_longest = 0
175
176	Variable/G root:Packages:NIST:Event:gEvent_tsdisp //Displayed slice
177	Variable/G root:Packages:NIST:Event:gEvent_nslices = 10 //Number of time slices
178
179	Variable/G root:Packages:NIST:Event:gEvent_logint = 1
180
181	Variable/G root:Packages:NIST:Event:gEvent_Mode = MODE_OSCILL // ==0 for "stream", ==1 for Oscillatory
182	Variable/G root:Packages:NIST:Event:gRemoveBadEvents = 1 // ==1 to remove "bad" events, ==0 to read "as-is"
183	Variable/G root:Packages:NIST:Event:gSortStreamEvents = 0 // ==1 to sort the event stream, a last resort for a stream of data
184
185	Variable/G root:Packages:NIST:Event:gEvent_ForceTmaxBin=1 //==1 to enforce t_longest in user-defined custom bins
186
187	NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
188
189
190	Make/D/O/N=(XBINS,YBINS,nslices) slicedData
191	Duplicate/O slicedData logslicedData
192	Duplicate/O slicedData dispsliceData
193
194
195	// for decimation
196	Variable/G root:Packages:NIST:Event:gEventFileTooLarge = 2500 // 2500 MB considered too large
197	Variable/G root:Packages:NIST:Event:gDecimation = 100
198	Variable/G root:Packages:NIST:Event:gEvent_t_longest_decimated = 0
199	Variable/G root:Packages:NIST:Event:gEvent_t_segment_start = 0
200	Variable/G root:Packages:NIST:Event:gEvent_t_segment_end = 0
201
202	// for large file splitting
203	String/G root:Packages:NIST:Event:gSplitFileList = "" // a list of the file names as split
204
205	// for editing
206	Variable/G root:Packages:NIST:Event:gStepTolerance = 5 // 5 = # of standard deviations from mean. See PutCursorsAtStep()
207
208	SetDataFolder root:
209	End
210
211	//
212	// -- extra bits of buttons... not used
213	//
214	// Button button9 title="Decimation",size={100,20},pos={490,400},proc=E_ShowDecimateButton
215	//
216	// Button button11,pos={490,245},size={150,20},proc=LoadDecimateButtonProc,title="Load and Decimate"
217	// Button button12,pos={490,277},size={150,20},proc=ConcatenateButtonProc,title="Concatenate"
218	// Button button13,pos={490,305},size={150,20},proc=DisplayConcatenatedButtonProc,title="Display Concatenated"
219	//
220	// GroupBox group0 title="Manual Controls",size={185,112},pos={490,220}
221	//
222	// NewPanel /W=(82,44,854,664)/N=EventModePanel/K=2
223	// DoWindow/C EventModePanel
224	// ModifyPanel fixedSize=1,noEdit =1
225	Proc EventModePanel()
226	PauseUpdate; Silent 1 // building window...
227	NewPanel /W=(82,44,884,664)/N=EventModePanel/K=2
228	DoWindow/C EventModePanel
229	// ModifyPanel fixedSize=1,noEdit =1
230
231	SetDrawLayer UserBack
232	DrawText 479,345,"Stream Data"
233	DrawLine 563,338,775,338
234	DrawText 479,419,"Oscillatory Data"
235	DrawLine 580,411,775,411
236
237	// ShowTools/A
238	Button button0,pos={14,87},size={150,20},proc=LoadEventLog_Button,title="Load Event Log File"
239	Button button0,fSize=12
240	TitleBox tb1,pos={475,500},size={266,86},fSize=10
241	TitleBox tb1,variable= root:Packages:NIST:Event:gEventDisplayString
242
243	CheckBox chkbox2,pos={376,151},size={81,15},proc=LogIntEvent_Proc,title="Log Intensity"
244	CheckBox chkbox2,fSize=10,variable= root:Packages:NIST:Event:gEvent_logint
245	CheckBox chkbox3,pos={14,125},size={119,15},title="Remove Bad Events?",fSize=10
246	CheckBox chkbox3,variable= root:Packages:NIST:Event:gRemoveBadEvents
247
248	Button doneButton,pos={738,36},size={50,20},proc=EventDone_Proc,title="Done"
249	Button doneButton,fSize=12
250	Button button2,pos={486,200},size={140,20},proc=ShowEventDataButtonProc,title="Show Event Data"
251	Button button3,pos={486,228},size={140,20},proc=ShowBinDetailsButtonProc,title="Show Bin Details"
252	Button button5,pos={633,228},size={140,20},proc=ExportSlicesButtonProc,title="Export Slices as VAX"
253	Button button6,pos={748,9},size={40,20},proc=EventModeHelpButtonProc,title="?"
254
255	Button button7,pos={211,33},size={120,20},proc=AdjustEventDataButtonProc,title="Adjust Events"
256	Button button8,pos={653,201},size={120,20},proc=CustomBinButtonProc,title="Custom Bins"
257	Button button4,pos={211,63},size={120,20},proc=UndoTimeSortButtonProc,title="Undo Time Sort"
258	Button button18,pos={211,90},size={120,20},proc=EC_ImportWavesButtonProc,title="Import Edited"
259
260	SetVariable setvar0,pos={208,149},size={160,16},proc=sliceSelectEvent_Proc,title="Display Time Slice"
261	SetVariable setvar0,fSize=10
262	SetVariable setvar0,limits={0,1000,1},value= root:Packages:NIST:Event:gEvent_tsdisp
263	SetVariable setvar1,pos={389,29},size={160,16},title="Number of slices",fSize=10
264	SetVariable setvar1,limits={1,1000,1},value= root:Packages:NIST:Event:gEvent_nslices
265	SetVariable setvar2,pos={389,54},size={160,16},title="Max Time (s)",fSize=10
266	SetVariable setvar2,value= root:Packages:NIST:Event:gEvent_t_longest
267
268	PopupMenu popup0,pos={389,77},size={119,20},proc=BinTypePopMenuProc,title="Bin Spacing"
269	PopupMenu popup0,fSize=10
270	PopupMenu popup0,mode=1,popvalue="Equal",value= #"\"Equal;Fibonacci;Custom;\""
271	Button button1,pos={389,103},size={120,20},fSize=12,proc=ProcessEventLog_Button,title="Bin Event Data"
272
273	Button button10,pos={488,305},size={100,20},proc=SplitFileButtonProc,title="Split Big File"
274	Button button14,pos={488,350},size={120,20},proc=Stream_LoadDecim,title="Load+Decimate"
275	Button button19,pos={639,350},size={130,20},proc=Stream_LoadAdjustedList,title="Load+Concatenate"
276	Button button20,pos={680,305},size={90,20},proc=ShowList_ToLoad,title="Show List"
277
278	Button button21,pos={649,378},size={120,20},proc=Stream_LoadAdjList_BinOnFly,title="Load+Accumulate"
279
280	SetVariable setvar3,pos={487,378},size={150,16},title="Decimation factor"
281	SetVariable setvar3,fSize=10
282	SetVariable setvar3,limits={1,inf,1},value= root:Packages:NIST:Event:gDecimation
283
284	Button button15_0,pos={488,425},size={110,20},proc=AccumulateSlicesButton,title="Add First Slice"
285	Button button16_1,pos={488,450},size={110,20},proc=AccumulateSlicesButton,title="Add Next Slice"
286	Button button17_2,pos={620,425},size={110,20},proc=AccumulateSlicesButton,title="Display Total"
287	Button button22,pos={620,450},size={120,20},proc=Osc_LoadAdjList_BinOnFly,title="Load+Accumulate"
288
289	CheckBox chkbox1_0,pos={25,34},size={69,14},title="Oscillatory",fSize=10
290	CheckBox chkbox1_0,mode=1,proc=EventModeRadioProc,value=1
291	CheckBox chkbox1_1,pos={25,59},size={53,14},title="Stream",fSize=10
292	CheckBox chkbox1_1,proc=EventModeRadioProc,value=0,mode=1
293	CheckBox chkbox1_2,pos={104,59},size={53,14},title="TISANE",fSize=10
294	CheckBox chkbox1_2,proc=EventModeRadioProc,value=0,mode=1
295	CheckBox chkbox1_3,pos={104,34},size={37,14},title="TOF",fSize=10
296	CheckBox chkbox1_3,proc=EventModeRadioProc,value=0,mode=1
297
298	GroupBox group0_0,pos={5,5},size={174,112},title="(1) Loading Mode",fSize=12,fStyle=1
299	GroupBox group0_1,pos={372,5},size={192,127},title="(3) Bin Events",fSize=12,fStyle=1
300	GroupBox group0_2,pos={477,169},size={310,92},title="(4) View / Export",fSize=12,fStyle=1
301	GroupBox group0_3,pos={191,5},size={165,117},title="(2) Edit Events",fSize=12,fStyle=1
302	GroupBox group0_4,pos={474,278},size={312,200},title="Split / Accumulate Files",fSize=12
303	GroupBox group0_4,fStyle=1
304
305	Display/W=(10,170,460,610)/HOST=#
306	AppendImage/T/G=1 :Packages:NIST:Event:dispsliceData // /G=1 flag prevents interpretation as RGB so 3, 4 slices display correctly
307	ModifyImage dispsliceData ctab= {,,ColdWarm,0}
308	ModifyImage dispsliceData ctabAutoscale=3
309	ModifyGraph margin(left)=14,margin(bottom)=14,margin(top)=14,margin(right)=14
310	ModifyGraph mirror=2
311	ModifyGraph nticks=4
312	ModifyGraph minor=1
313	ModifyGraph fSize=9
314	ModifyGraph standoff=0
315	ModifyGraph tkLblRot(left)=90
316	ModifyGraph btLen=3
317	ModifyGraph tlOffset=-2
318	RenameWindow #,Event_slicegraph
319	SetActiveSubwindow ##
320	EndMacro
321
322
323
324
325	// mode selector
326	//Static Constant MODE_STREAM = 0
327	//Static Constant MODE_OSCILL = 1
328	//Static Constant MODE_TISANE = 2
329	//Static Constant MODE_TOF = 3
330	//
331	Function EventModeRadioProc(name,value)
332	String name
333	Variable value
334
335	NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
336
337	strswitch (name)
338	case "chkbox1_0":
339	gEventModeRadioVal= MODE_OSCILL
340	break
341	case "chkbox1_1":
342	gEventModeRadioVal= MODE_STREAM
343	break
344	case "chkbox1_2":
345	gEventModeRadioVal= MODE_TISANE
346	break
347	case "chkbox1_3":
348	gEventModeRadioVal= MODE_TOF
349	break
350	endswitch
351	CheckBox chkbox1_0,value= gEventModeRadioVal==MODE_OSCILL
352	CheckBox chkbox1_1,value= gEventModeRadioVal==MODE_STREAM
353	CheckBox chkbox1_2,value= gEventModeRadioVal==MODE_TISANE
354	CheckBox chkbox1_3,value= gEventModeRadioVal==MODE_TOF
355
356	return(0)
357	End
358
359	Function AdjustEventDataButtonProc(ba) : ButtonControl
360	STRUCT WMButtonAction &ba
361
362	switch( ba.eventCode )
363	case 2: // mouse up
364	// click code here
365	Execute "ShowEventCorrectionPanel()"
366	//
367	break
368	case -1: // control being killed
369	break
370	endswitch
371
372	return 0
373	End
374
375	Function CustomBinButtonProc(ba) : ButtonControl
376	STRUCT WMButtonAction &ba
377
378	switch( ba.eventCode )
379	case 2: // mouse up
380	// click code here
381	Execute "Show_CustomBinPanel()"
382	//
383	break
384	case -1: // control being killed
385	break
386	endswitch
387
388	return 0
389	End
390
391
392	Function ShowEventDataButtonProc(ba) : ButtonControl
393	STRUCT WMButtonAction &ba
394
395	switch( ba.eventCode )
396	case 2: // mouse up
397	// click code here
398	Execute "ShowRescaledTimeGraph()"
399	//
400	DifferentiatedTime()
401	//
402	break
403	case -1: // control being killed
404	break
405	endswitch
406
407	return 0
408	End
409
410	Function BinTypePopMenuProc(pa) : PopupMenuControl
411	STRUCT WMPopupAction &pa
412
413	switch( pa.eventCode )
414	case 2: // mouse up
415	Variable popNum = pa.popNum
416	String popStr = pa.popStr
417	if(cmpstr(popStr,"Custom")==0)
418	Execute "Show_CustomBinPanel()"
419	endif
420	break
421	case -1: // control being killed
422	break
423	endswitch
424
425	return 0
426	End
427
428	Function ShowBinDetailsButtonProc(ba) : ButtonControl
429	STRUCT WMButtonAction &ba
430
431	switch( ba.eventCode )
432	case 2: // mouse up
433	// click code here
434	Execute "ShowBinTable()"
435	Execute "BinEventBarGraph()"
436	break
437	case -1: // control being killed
438	break
439	endswitch
440
441	return 0
442	End
443
444	Function UndoTimeSortButtonProc(ba) : ButtonControl
445	STRUCT WMButtonAction &ba
446
447	switch( ba.eventCode )
448	case 2: // mouse up
449	// click code here
450	Execute "UndoTheSorting()"
451	break
452	case -1: // control being killed
453	break
454	endswitch
455
456	return 0
457	End
458
459	Function ExportSlicesButtonProc(ba) : ButtonControl
460	STRUCT WMButtonAction &ba
461
462	switch( ba.eventCode )
463	case 2: // mouse up
464	// click code here
465	Execute "ExportSlicesAsVAX()" //will invoke the dialog
466	break
467	case -1: // control being killed
468	break
469	endswitch
470
471	return 0
472	End
473
474	Function EventModeHelpButtonProc(ba) : ButtonControl
475	STRUCT WMButtonAction &ba
476
477	switch( ba.eventCode )
478	case 2: // mouse up
479	// click code here
480	DisplayHelpTopic/Z "Event Mode Data"
481	break
482	case -1: // control being killed
483	break
484	endswitch
485
486	return 0
487	End
488
489
490	Function EventDone_Proc(ba) : ButtonControl
491	STRUCT WMButtonAction &ba
492
493	String win = ba.win
494	switch (ba.eventCode)
495	case 2:
496	DoWindow/K EventModePanel
497	break
498	endswitch
499	return(0)
500	End
501
502
503
504	Function ProcessEventLog_Button(ctrlName) : ButtonControl
505	String ctrlName
506
507	NVAR mode=root:Packages:NIST:Event:gEvent_Mode
508
509	if(mode == MODE_STREAM)
510	Stream_ProcessEventLog("")
511	endif
512
513	if(mode == MODE_OSCILL)
514	Osc_ProcessEventLog("")
515	endif
516
517	// If TOF mode, process as Oscillatory -- that is, take the times as is
518	if(mode == MODE_TOF)
519	Osc_ProcessEventLog("")
520	endif
521
522	// toggle the checkbox for log display to force the display to be correct
523	NVAR gLog = root:Packages:NIST:Event:gEvent_logint
524	LogIntEvent_Proc("",gLog)
525
526	return(0)
527	end
528
529	// for oscillatory mode
530	//
531	Function Osc_ProcessEventLog(ctrlName)
532	String ctrlName
533
534	Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
535
536	Wave binnedData = root:Packages:NIST:Event:binnedData
537	Wave xLoc = root:Packages:NIST:Event:xLoc
538	Wave yLoc = root:Packages:NIST:Event:yLoc
539
540	// now with the number of slices and max time, process the events
541
542	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
543	NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
544
545	SetDataFolder root:Packages:NIST:Event //don't count on the folder remaining here
546
547	Make/D/O/N=(XBINS,YBINS,nslices) slicedData
548
549	Wave slicedData = slicedData
550	Wave rescaledTime = rescaledTime
551	Wave timePt = timePt
552	Make/O/D/N=(128,128) tmpData
553	Make/O/D/N=(nslices+1) binEndTime,binCount
554	Make/O/D/N=(nslices) timeWidth
555	Wave timeWidth = timeWidth
556	Wave binEndTime = binEndTime
557	Wave binCount = binCount
558
559	variable ii,del,p1,p2,t1,t2
560	del = t_longest/nslices
561
562	slicedData = 0
563	binEndTime[0]=0
564	BinCount[nslices]=0
565
566
567	String binTypeStr=""
568	ControlInfo /W=EventModePanel popup0
569	binTypeStr = S_value
570
571	strswitch(binTypeStr) // string switch
572	case "Equal": // execute if case matches expression
573	SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
574	break // exit from switch
575	case "Fibonacci": // execute if case matches expression
576	SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
577	break
578	case "Log": // execute if case matches expression
579	SetLogBins(binEndTime,timeWidth,nslices,t_longest)
580	break
581	case "Custom": // execute if case matches expression
582	//bins are set by the user on the panel - assume it's good to go
583	break
584	default: // optional default expression executed
585	DoAlert 0,"No match for bin type, Equal bins used"
586	SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
587	endswitch
588
589
590	// now before binning, sort the data
591
592	//MakeIndex is REALLY SLOW - this is slow - to allow undoing the sorting and starting over, but if you don't,
593	// you'll never be able to undo the sort
594	//
595	//
596	// for a 500 MB hst, MakeIndex = 83s, everything else in this function totalled < 5s
597	//
598	SetDataFolder root:Packages:NIST:Event:
599
600	if(WaveExists($"root:Packages:NIST:Event:OscSortIndex") == 0 )
601	Duplicate/O rescaledTime OscSortIndex
602	// tic()
603	MakeIndex rescaledTime OscSortIndex
604	// toc()
605	// tic()
606	IndexSort OscSortIndex, yLoc,xLoc,timePt,rescaledTime
607	// toc()
608	//SetDataFolder root:Packages:NIST:Event
609	// tic()
610	IndexForHistogram(xLoc,yLoc,binnedData) // index the events AFTER sorting
611	// toc()
612	//SetDataFolder root:
613	Endif
614
615	Wave index = root:Packages:NIST:Event:SavedIndex //this is the histogram index
616
617	// tic()
618	for(ii=0;ii<nslices;ii+=1)
619	if(ii==0)
620	// t1 = ii*del
621	// t2 = (ii+1)*del
622	p1 = BinarySearch(rescaledTime,0)
623	p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
624	else
625	// t2 = (ii+1)*del
626	p1 = p2+1 //one more than the old one
627	p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
628	endif
629
630	// typically zero will never be a valid time value in oscillatory mode. in "stream" mode, the first is normalized to == 0
631	// but not here - times are what they are.
632	if(p1 == -1)
633	Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
634	p1 = 0 //set to the first point if it's off the end
635	Endif
636
637	if(p2 == -2)
638	Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
639	p2 = numpnts(rescaledTime)-1 //set to the last point if it's off the end
640	Endif
641	// Print p1,p2
642
643
644	tmpData=0
645	JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
646	slicedData[][][ii] = tmpData[p][q]
647
648	// binEndTime[ii+1] = t2
649	binCount[ii] = sum(tmpData,-inf,inf)
650	endfor
651
652	// toc()
653	Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
654	Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
655	logslicedData = log(slicedData)
656
657	SetDataFolder root:
658	return(0)
659	End
660
661	// for a "continuous exposure"
662	//
663	// if there is a sort of these events, I need to re-index the events for the histogram
664	// - see the oscillatory mode - and sort the events here, then immediately re-index for the histogram
665	// - but with the added complication that I need to always remember to index for the histogram, every time
666	// - since I don't know if I've sorted or un-sorted. Osc mode always forces a re-sort and a re-index
667	//
668	Function Stream_ProcessEventLog(ctrlName)
669	String ctrlName
670
671	// NVAR slicewidth = root:Packages:NIST:gTISANE_slicewidth
672
673
674	Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
675
676	Wave binnedData = root:Packages:NIST:Event:binnedData
677	Wave xLoc = root:Packages:NIST:Event:xLoc
678	Wave yLoc = root:Packages:NIST:Event:yLoc
679
680	// now with the number of slices and max time, process the events
681
682	NVAR yesSortStream = root:Packages:NIST:Event:gSortStreamEvents //do I sort the events?
683	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
684	NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
685
686	SetDataFolder root:Packages:NIST:Event //don't count on the folder remaining here
687
688	Make/D/O/N=(XBINS,YBINS,nslices) slicedData
689
690	Wave slicedData = slicedData
691	Wave rescaledTime = rescaledTime
692	Make/O/D/N=(128,128) tmpData
693	Make/O/D/N=(nslices+1) binEndTime,binCount//,binStartTime
694	Make/O/D/N=(nslices) timeWidth
695	Wave binEndTime = binEndTime
696	Wave timeWidth = timeWidth
697	Wave binCount = binCount
698
699	variable ii,del,p1,p2,t1,t2
700	del = t_longest/nslices
701
702	slicedData = 0
703	binEndTime[0]=0
704	BinCount[nslices]=0
705
706	String binTypeStr=""
707	ControlInfo /W=EventModePanel popup0
708	binTypeStr = S_value
709
710	strswitch(binTypeStr) // string switch
711	case "Equal": // execute if case matches expression
712	SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
713	break // exit from switch
714	case "Fibonacci": // execute if case matches expression
715	SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
716	break
717	case "Log": // execute if case matches expression
718	SetLogBins(binEndTime,timeWidth,nslices,t_longest)
719	break
720	case "Custom": // execute if case matches expression
721	//bins are set by the user on the panel - assume it's good to go
722	break
723	default: // optional default expression executed
724	DoAlert 0,"No match for bin type, Equal bins used"
725	SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
726	endswitch
727
728	// TODO
729	// the global exists for this switch, but it is not implemented - not sure whether
730	// it's correct to implement this at all --
731	//
732	if(yesSortStream == 1)
733	SortTimeData()
734	endif
735
736	// index the events before binning
737	// if there is a sort of these events, I need to re-index the events for the histogram
738	// SetDataFolder root:Packages:NIST:Event
739	IndexForHistogram(xLoc,yLoc,binnedData)
740	// SetDataFolder root:
741	Wave index = root:Packages:NIST:Event:SavedIndex //the index for the histogram
742
743
744	for(ii=0;ii<nslices;ii+=1)
745	if(ii==0)
746	// t1 = ii*del
747	// t2 = (ii+1)*del
748	p1 = BinarySearch(rescaledTime,0)
749	p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
750	else
751	// t2 = (ii+1)*del
752	p1 = p2+1 //one more than the old one
753	p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
754	endif
755
756	if(p1 == -1)
757	Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
758	p1 = 0 //set to the first point if it's off the end
759	Endif
760	if(p2 == -2)
761	Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
762	p2 = numpnts(rescaledTime)-1 //set to the last point if it's off the end
763	Endif
764	// Print p1,p2
765
766
767	tmpData=0
768	JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
769	slicedData[][][ii] = tmpData[p][q]
770
771	// binEndTime[ii+1] = t2
772	binCount[ii] = sum(tmpData,-inf,inf)
773	endfor
774
775	Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
776	Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
777	logslicedData = log(slicedData)
778
779	SetDataFolder root:
780	return(0)
781	End
782
783
784	Proc UndoTheSorting()
785	Osc_UndoSort()
786	End
787
788	// for oscillatory mode
789	//
790	// -- this takes the previously generated index, and un-sorts the data to restore to the
791	// "as-collected" state
792	//
793	Function Osc_UndoSort()
794
795	SetDataFolder root:Packages:NIST:Event //don't count on the folder remaining here
796	Wave rescaledTime = rescaledTime
797	Wave OscSortIndex = OscSortIndex
798	Wave yLoc = yLoc
799	Wave xLoc = xLoc
800	Wave timePt = timePt
801
802	Sort OscSortIndex OscSortIndex,yLoc,xLoc,timePt,rescaledTime
803
804	KillWaves/Z OscSortIndex
805
806	SetDataFolder root:
807	return(0)
808	End
809
810
811	// now before binning, sort the data
812	//
813	//this is slow - undoing the sorting and starting over, but if you don't,
814	// you'll never be able to undo the sort
815	//
816	Function SortTimeData()
817
818
819	SetDataFolder root:Packages:NIST:Event:
820
821	KillWaves/Z OscSortIndex
822
823	if(WaveExists($"root:Packages:NIST:Event:OscSortIndex") == 0 )
824	Duplicate/O rescaledTime OscSortIndex
825	MakeIndex rescaledTime OscSortIndex
826	IndexSort OscSortIndex, yLoc,xLoc,timePt,rescaledTime
827	Endif
828
829	SetDataFolder root:
830	return(0)
831	End
832
833
834
835	Function SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
836	Wave binEndTime,timeWidth
837	Variable nslices,t_longest
838
839	Variable del,ii,t2
840	binEndTime[0]=0 //so the bar graph plots right...
841	del = t_longest/nslices
842
843	for(ii=0;ii<nslices;ii+=1)
844	t2 = (ii+1)*del
845	binEndTime[ii+1] = t2
846	endfor
847	binEndTime[ii+1] = t_longest*(1-1e-6) //otherwise floating point errors such that the last time point is off the end of the Binary search
848
849	timeWidth = binEndTime[p+1]-binEndTime[p]
850
851	return(0)
852	End
853
854	// TODO
855	// either get this to work, or scrap it entirely. it currently isn't on the popup
856	// so it can't be accessed
857	Function SetLogBins(binEndTime,timeWidth,nslices,t_longest)
858	Wave binEndTime,timeWidth
859	Variable nslices,t_longest
860
861	Variable tMin,ii
862
863	Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
864
865	binEndTime[0]=0 //so the bar graph plots right...
866
867	// just like the log-scaled q-points
868	tMin = rescaledTime[1]/1 //just a guess... can't use tMin=0, and rescaledTime[0] == 0 by definition
869	Print rescaledTime[1], tMin
870	for(ii=0;ii<nslices;ii+=1)
871	binEndTime[ii+1] =alog(log(tMin) + (ii+1)*((log(t_longest)-log(tMin))/nslices))
872	endfor
873	binEndTime[ii+1] = t_longest //otherwise floating point errors such that the last time point is off the end of the Binary search
874
875	timeWidth = binEndTime[p+1]-binEndTime[p]
876
877	return(0)
878	End
879
880	Function MakeFibonacciWave(w,num)
881	Wave w
882	Variable num
883
884	//skip the initial zero
885	Variable f1,f2,ii
886	f1=1
887	f2=1
888	w[0] = f1
889	w[1] = f2
890	for(ii=2;ii<num;ii+=1)
891	w[ii] = f1+f2
892	f1=f2
893	f2=w[ii]
894	endfor
895
896	return(0)
897	end
898
899	Function SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
900	Wave binEndTime,timeWidth
901	Variable nslices,t_longest
902
903	Variable tMin,ii,total,t2,tmp
904	Make/O/D/N=(nslices) fibo
905	fibo=0
906	MakeFibonacciWave(fibo,nslices)
907
908	// Make/O/D tmpFib={1,1,2,3,5,8,13,21,34,55,89,144,233,377,610,987,1597,2584,4181,6765,10946}
909
910	binEndTime[0]=0 //so the bar graph plots right...
911	total = sum(fibo,0,nslices-1) //total number of "pieces"
912
913	tmp=0
914	for(ii=0;ii<nslices;ii+=1)
915	t2 = sum(fibo,0,ii)/total*t_longest
916	binEndTime[ii+1] = t2
917	endfor
918	binEndTime[ii+1] = t_longest //otherwise floating point errors such that the last time point is off the end of the Binary search
919
920	timeWidth = binEndTime[p+1]-binEndTime[p]
921
922	return(0)
923	End
924
925
926
927	// TODO:
928	//
929	// ** currently, the "stream" loader uses the first data point as time=0
930	// and rescales everything to that time. "Osc" loading uses the times "as-is"
931	// from the file, trusting the times to be correct.
932	//
933	// Would TISANE or TOF need a different loader?
934	//
935	Function LoadEventLog_Button(ctrlName) : ButtonControl
936	String ctrlName
937
938	NVAR mode=root:Packages:NIST:Event:gEvent_mode
939	Variable err=0
940	Variable fileref,totBytes
941	NVAR fileTooLarge = root:Packages:NIST:Event:gEventFileTooLarge //limit load to 150MB
942
943	SVAR filename = root:Packages:NIST:Event:gEvent_logfile
944	NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
945	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
946
947	String fileFilters = "All Files:.;Data Files (.txt):.txt;"
948	String abortStr
949
950	PathInfo/S catPathName
951	if(V_flag==0)
952	DoAlert 0,"Please 'Pick Path' to the data from the Main (yellow) Panel."
953	return(0)
954	endif
955
956
957	Open/R/D/P=catPathName/F=fileFilters fileref
958	filename = S_filename
959	if(strlen(S_filename) == 0)
960	// user cancelled
961	DoAlert 0,"No file selected, no file loaded."
962	return(1)
963	endif
964
965	/// Abort if the files are too large
966	Open/R fileref as fileName
967	FStatus fileref
968	Close fileref
969
970	totBytes = V_logEOF/1e6 //in MB
971	if(totBytes > fileTooLarge)
972	sprintf abortStr,"File is %g MB, larger than the limit of %g MB. Split and Decimate.",totBytes,fileTooLarge
973	Abort abortStr
974	endif
975
976
977	#if (exists("EventLoadWave")==4)
978	LoadEvents_XOP()
979	#else
980	LoadEvents()
981	#endif
982
983	SetDataFolder root:Packages:NIST:Event:
984
985	//tic()
986	Wave timePt=timePt
987	Wave xLoc=xLoc
988	Wave yLoc=yLoc
989	CleanupTimes(xLoc,yLoc,timePt) //remove zeroes
990	//toc()
991
992
993	/////
994	// now do a little processing of the times based on the type of data
995	//
996	if(mode == MODE_STREAM) // continuous "Stream" mode - start from zero
997	Duplicate/O timePt rescaledTime
998	rescaledTime = 1e-7*(timePt-timePt[0]) //convert to seconds and start from zero
999	t_longest = waveMax(rescaledTime) //should be the last point
1000	endif
1001
1002	if(mode == MODE_OSCILL) // oscillatory mode - don't adjust the times, we get periodic t0 to reset t=0
1003	Duplicate/O timePt rescaledTime
1004	rescaledTime *= 1e-7 //convert to seconds and that's all
1005	t_longest = waveMax(rescaledTime) //if oscillatory, won't be the last point, so get it this way
1006
1007	KillWaves/Z OscSortIndex //to make sure that there is no old index hanging around
1008	endif
1009
1010	// MODE_TISANE
1011	//TODO -- tisane doesn't do anything different than oscillatory. Is this really correct??
1012	if(mode == MODE_TISANE) // TISANE mode - don't adjust the times, we get periodic t0 to reset t=0
1013	Duplicate/O timePt rescaledTime
1014	rescaledTime *= 1e-7 //convert to seconds and that's all
1015	t_longest = waveMax(rescaledTime) //if oscillatory, won't be the last point, so get it this way
1016
1017	KillWaves/Z OscSortIndex //to make sure that there is no old index hanging around
1018	endif
1019
1020	// MODE_TOF
1021	if(mode == MODE_TOF) // TOF mode - don't adjust the times, we get periodic t0 to reset t=0
1022	Duplicate/O timePt rescaledTime
1023	rescaledTime *= 1e-7 //convert to seconds and that's all
1024	t_longest = waveMax(rescaledTime) //if oscillatory, won't be the last point, so get it this way
1025
1026	KillWaves/Z OscSortIndex //to make sure that there is no old index hanging around
1027	endif
1028
1029	SetDataFolder root:
1030
1031	STRUCT WMButtonAction ba
1032	ba.eventCode = 2
1033	ShowEventDataButtonProc(ba)
1034
1035	return(0)
1036	End
1037
1038	//// for the mode of "one continuous exposure"
1039	////
1040	//Function Stream_LoadEventLog(ctrlName)
1041	// String ctrlName
1042	//
1043	// Variable fileref
1044	//
1045	// SVAR filename = root:Packages:NIST:Event:gEvent_logfile
1046	// NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
1047	// NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1048	//
1049	// String fileFilters = "All Files:.;Data Files (.txt):.txt;"
1050	//
1051	// Open/R/D/F=fileFilters fileref
1052	// filename = S_filename
1053	// if(strlen(S_filename) == 0)
1054	// // user cancelled
1055	// DoAlert 0,"No file selected, no file loaded."
1056	// return(1)
1057	// endif
1058	//
1059	//#if (exists("EventLoadWave")==4)
1060	// LoadEvents_XOP()
1061	//#else
1062	// LoadEvents()
1063	//#endif
1064	//
1065	// SetDataFolder root:Packages:NIST:Event:
1066	//
1067	////tic()
1068	// Wave timePt=timePt
1069	// Wave xLoc=xLoc
1070	// Wave yLoc=yLoc
1071	// CleanupTimes(xLoc,yLoc,timePt) //remove zeroes
1072	//
1073	////toc()
1074	//
1075	// Duplicate/O timePt rescaledTime
1076	// rescaledTime = 1e-7*(timePt-timePt[0]) //convert to seconds and start from zero
1077	// t_longest = waveMax(rescaledTime) //should be the last point
1078	//
1079	// SetDataFolder root:
1080	//
1081	// return(0)
1082	//End
1083	//
1084	//// for the mode "oscillatory"
1085	////
1086	//Function Osc_LoadEventLog(ctrlName)
1087	// String ctrlName
1088	//
1089	// Variable fileref
1090	//
1091	// SVAR filename = root:Packages:NIST:Event:gEvent_logfile
1092	// NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
1093	// NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1094	//
1095	// String fileFilters = "All Files:.;Data Files (.txt):.txt;"
1096	//
1097	// Open/R/D/F=fileFilters fileref
1098	// filename = S_filename
1099	// if(strlen(S_filename) == 0)
1100	// // user cancelled
1101	// DoAlert 0,"No file selected, no file loaded."
1102	// return(1)
1103	// endif
1104	//
1105	//#if (exists("EventLoadWave")==4)
1106	// LoadEvents_XOP()
1107	//#else
1108	// LoadEvents()
1109	//#endif
1110	//
1111	// SetDataFolder root:Packages:NIST:Event:
1112	//
1113	// Wave timePt=timePt
1114	// Wave xLoc=xLoc
1115	// Wave yLoc=yLoc
1116	// CleanupTimes(xLoc,yLoc,timePt) //remove zeroes
1117	//
1118	// Duplicate/O timePt rescaledTime
1119	// rescaledTime *= 1e-7 //convert to seconds and that's all
1120	// t_longest = waveMax(rescaledTime) //if oscillatory, won't be the last point, so get it this way
1121	//
1122	// KillWaves/Z OscSortIndex //to make sure that there is no old index hanging around
1123	//
1124	// SetDataFolder root:
1125	//
1126	// return(0)
1127	//End
1128
1129
1130	//
1131	// -- MUCH faster to count the number of lines to remove, then delete (N)
1132	// rather then delete them one-by-one in the do-loop
1133	Function CleanupTimes(xLoc,yLoc,timePt)
1134	Wave xLoc,yLoc,timePt
1135
1136	// start at the back and remove zeros
1137	Variable num=numpnts(xLoc),ii,numToRemove
1138
1139	numToRemove = 0
1140	ii=num
1141	do
1142	ii -= 1
1143	if(timePt[ii] == 0 && xLoc[ii] == 0 && yLoc[ii] == 0)
1144	numToRemove += 1
1145	endif
1146	while(timePt[ii-1] == 0 && xLoc[ii-1] == 0 && yLoc[ii-1] == 0)
1147
1148	if(numToRemove != 0)
1149	DeletePoints ii, numToRemove, xLoc,yLoc,timePt
1150	endif
1151
1152	return(0)
1153	End
1154
1155	Function LogIntEvent_Proc(ctrlName,checked) : CheckBoxControl
1156	String ctrlName
1157	Variable checked
1158
1159	SetDataFolder root:Packages:NIST:Event
1160
1161	Wave slicedData = slicedData
1162	Wave logSlicedData = logSlicedData
1163	Wave dispSliceData = dispSliceData
1164
1165	if(checked)
1166	logslicedData = log(slicedData)
1167	Duplicate/O logslicedData dispsliceData
1168	else
1169	Duplicate/O slicedData dispsliceData
1170	endif
1171
1172	NVAR selectedslice = root:Packages:NIST:Event:gEvent_tsdisp
1173
1174	sliceSelectEvent_Proc("", selectedslice, "", "")
1175
1176	SetDataFolder root:
1177
1178	End
1179
1180
1181	// TODO (DONE)
1182	// this "fails" for data sets that have 3 or 4 slices, as the ModifyImage command
1183	// interprets the data as being RGB - and so does nothing.
1184	// need to find a way around this
1185	//
1186	//// When first plotted, AppendImage/G=1 flag prevents interpretation as RGB so 3, 4 slices display correctly
1187	///
1188	// I could modify this procedure to use the log = 0\|1 keyword for the log Z display
1189	// rather than creating a duplicate wave of log(data)
1190	//
1191	Function sliceSelectEvent_Proc(ctrlName, varNum, varStr, varName) : SetVariableControl
1192	String ctrlName
1193	Variable varNum
1194	String varStr
1195	String varName
1196
1197	NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
1198	NVAR selectedslice = root:Packages:NIST:Event:gEvent_tsdisp
1199
1200	if(varNum < 0)
1201	selectedslice = 0
1202	DoUpdate
1203	elseif (varNum > nslices-1)
1204	selectedslice = nslices-1
1205	DoUpdate
1206	else
1207	ModifyImage/W=EventModePanel#Event_slicegraph ''#0 plane = varNum
1208	endif
1209
1210	End
1211
1212	Function DifferentiatedTime()
1213
1214	Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
1215
1216	SetDataFolder root:Packages:NIST:Event:
1217
1218	Differentiate rescaledTime/D=rescaledTime_DIF
1219	// Display rescaledTime,rescaledTime_DIF
1220	DoWindow/F Differentiated_Time
1221	if(V_flag == 0)
1222	Display/N=Differentiated_Time/K=1 rescaledTime_DIF
1223	Legend
1224	Modifygraph gaps=0
1225	ModifyGraph zero(left)=1
1226	Label left "\\Z14Delta (dt/event)"
1227	Label bottom "\\Z14Event number"
1228	endif
1229
1230	SetDataFolder root:
1231
1232	return(0)
1233	End
1234
1235
1236	//
1237	// for the bit shifts, see the decimal-binary conversion
1238	// http://www.binaryconvert.com/convert_unsigned_int.html
1239	//
1240	// K0 = 536870912
1241	// Print (K0 & 0x08000000)/134217728 //bit 27 only, shift by 2^27
1242	// Print (K0 & 0x10000000)/268435456 //bit 28 only, shift by 2^28
1243	// Print (K0 & 0x20000000)/536870912 //bit 29 only, shift by 2^29
1244	//
1245	// This is duplicated by the XOP, but the Igor code allows quick access to print out
1246	// all of the gory details of the events and every little bit of them. the print
1247	// statements and flags are kept for this reason, so the code is a bit messy.
1248	//
1249	Function LoadEvents()
1250
1251	NVAR time_msw = root:Packages:NIST:Event:gEvent_time_msw
1252	NVAR time_lsw = root:Packages:NIST:Event:gEvent_time_lsw
1253	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1254
1255	SVAR filepathstr = root:Packages:NIST:Event:gEvent_logfile
1256	SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
1257
1258
1259	//// Variable decFac = 10 //decimation factor
1260	//// Variable jj,keep
1261
1262	SetDataFolder root:Packages:NIST:Event
1263
1264	Variable fileref
1265	String buffer
1266	String fileStr,tmpStr
1267	Variable dataval,timeval,type,numLines,verbose,verbose3
1268	Variable xval,yval,rollBit,nRoll,roll_time,bit29,bit28,bit27
1269	Variable ii,flaggedEvent,rolloverHappened,numBad=0,tmpPP=0,tmpT0=0
1270	Variable Xmax, yMax
1271
1272	xMax = 127 // number the detector from 0->127
1273	yMax = 127
1274
1275	verbose3 = 0 //prints out the rollover events (type==3)
1276	verbose = 0
1277	numLines = 0
1278
1279
1280	// what I really need is the number of XY events
1281	Variable numXYevents,num1,num2,num3,num0,totBytes,numPP,numT0,numDL,numFF,numZero
1282	Variable numRemoved
1283	numXYevents = 0
1284	num0 = 0
1285	num1 = 0
1286	num2 = 0
1287	num3 = 0
1288	numPP = 0
1289	numT0 = 0
1290	numDL = 0
1291	numFF = 0
1292	numZero = 0
1293	numRemoved = 0
1294
1295	//tic()
1296	Open/R fileref as filepathstr
1297	FStatus fileref
1298	Close fileref
1299
1300	totBytes = V_logEOF
1301	Print "total bytes = ", totBytes
1302	if(EventDataType(filePathStr) != 1) //if not hst extension of raw data
1303	Abort "Data must be raw event data with file extension .hst for this operation"
1304	return(0)
1305	endif
1306	//toc()
1307	//
1308
1309
1310	// do a "pre-scan to get some of the counts, so that I can allocate space. This does
1311	// double the read time, but is still faster than adding points to waves as the file is read
1312	//
1313
1314	tic()
1315
1316	Open/R fileref as filepathstr
1317	do
1318	do
1319	FReadLine fileref, buffer //skip the "blank" lines that have one character
1320	while(strlen(buffer) == 1)
1321
1322	if (strlen(buffer) == 0)
1323	break
1324	endif
1325
1326	sscanf buffer,"%x",dataval
1327
1328	// two most sig bits (31-30)
1329	type = (dataval & 0xC0000000)/1073741824 //right shift by 2^30
1330
1331	if(type == 0)
1332	num0 += 1
1333	numXYevents += 1
1334	endif
1335	if(type == 2)
1336	num2 += 1
1337	numXYevents += 1
1338	endif
1339	if(type == 1)
1340	num1 += 1
1341	endif
1342	if(type == 3)
1343	num3 += 1
1344	endif
1345
1346	bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1347
1348	if(type==0 \|\| type==2)
1349	numPP += round(bit29)
1350	endif
1351
1352	if(type==1 \|\| type==3)
1353	numT0 += round(bit29)
1354	endif
1355
1356	if(dataval == 0)
1357	numZero += 1
1358	endif
1359
1360	while(1)
1361	Close fileref
1362	// done counting the number of XY events
1363	printf("Igor pre-scan done in ")
1364	toc()
1365
1366
1367	Print "(Igor) numT0 = ",numT0
1368	Print "num0 = ",num0
1369	Print "num1 = ",num1
1370	Print "num2 = ",num2
1371	Print "num3 = ",num3
1372
1373	//
1374	//
1375	// Printf "numXYevents = %d\r",numXYevents
1376	// Printf "XY = num0 = %d\r",num0
1377	// Printf "XY time = num2 = %d\r",num2
1378	// Printf "time MSW = num1 = %d\r",num1
1379	// Printf "Rollover = num3 = %d\r",num3
1380	// Printf "num0 + num2 = %d\r",num0+num2
1381
1382	// dispStr will be displayed on the panel
1383	fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
1384
1385	sprintf tmpStr, "%s: %d total bytes\r",fileStr,totBytes
1386	dispStr = tmpStr
1387	sprintf tmpStr,"numXYevents = %d\r",numXYevents
1388	dispStr += tmpStr
1389	sprintf tmpStr,"PP = %d : ",numPP
1390	dispStr += tmpStr
1391	sprintf tmpStr,"ZeroData = %d\r",numZero
1392	dispStr += tmpStr
1393	sprintf tmpStr,"Rollover = %d",num3
1394	dispStr += tmpStr
1395
1396	// /I/U is unsigned 32-bit integer (for the time)
1397	// /B/U is unsigned 8-bit integer (max val=255) for the x and y values
1398
1399	Make/O/U/N=(numXYevents) xLoc,yLoc
1400	Make/O/D/N=(numXYevents) timePt
1401	//// Make/O/U/N=(numXYevents/decFac) xLoc,yLoc
1402	//// Make/O/D/N=(numXYevents/decFac) timePt
1403	// Make/O/U/N=(totBytes/4) xLoc,yLoc //too large, trim when done (bad idea)
1404	// Make/O/D/N=(totBytes/4) timePt
1405	Make/O/D/N=1000 badTimePt,badEventNum,PPTime,PPEventNum,T0Time,T0EventNum
1406	badTimePt=0
1407	badEventNum=0
1408	PPTime=0
1409	PPEventNum=0
1410	T0Time=0
1411	T0EventNum=0
1412	xLoc=0
1413	yLoc=0
1414	timePt=0
1415
1416	nRoll = 0 //number of rollover events
1417	roll_time = 2^26 //units of 10-7 sec
1418
1419	NVAR removeBadEvents = root:Packages:NIST:Event:gRemoveBadEvents
1420
1421	time_msw=0
1422
1423	tic()
1424
1425	ii = 0 //indexes the points in xLoc,yLoc,timePt
1426	//// keep = decFac //keep the first point
1427
1428
1429	Open/R fileref as filepathstr
1430
1431	// remove events at the beginning up to a type==2 so that the msw and lsw times are reset properly
1432	if(RemoveBadEvents == 1)
1433	do
1434	do
1435	FReadLine fileref, buffer //skip the "blank" lines that have one character
1436	while(strlen(buffer) == 1)
1437
1438	if (strlen(buffer) == 0)
1439	break
1440	endif
1441
1442	sscanf buffer,"%x",dataval
1443	// two most sig bits (31-30)
1444	type = (dataval & 0xC0000000)/1073741824 //right shift by 2^30
1445
1446	if(type == 2)
1447	// this is the first event with a proper time value, so process the XY-time event as ususal
1448	// and then break to drop to the main loop, where the next event == type 1
1449
1450	xval = xMax - (dataval & 255) //last 8 bits (7-0)
1451	yval = (dataval & 65280)/256 //bits 15-8, right shift by 2^8
1452
1453	time_lsw = (dataval & 536805376)/65536 //13 bits, 28-16, right shift by 2^16
1454
1455	if(verbose)
1456	// printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1457	printf "%u : %u : %u : %u\r",dataval,timeval,xval,yval
1458	endif
1459
1460	// this is the first point, be sure that ii = 0, and always keep this point
1461	//// if(keep==decFac)
1462	ii = 0
1463	xLoc[ii] = xval
1464	yLoc[ii] = yval
1465	//// keep = 0
1466	//// endif
1467	Print "At beginning of file, numBad = ",numBad
1468	break // the next do loop processes the bulk of the file (** the next event == type 1 = MIR)
1469	else
1470	numBad += 1
1471	numRemoved += 1
1472	endif
1473
1474	//ii+=1 don't increment the counter
1475	while(1)
1476	endif
1477
1478	// now read the main portion of the file.
1479	//// // keep is = 0 if bad points were removed, or is decFac is I need to keep the first point
1480	do
1481	do
1482	FReadLine fileref, buffer //skip the "blank" lines that have one character
1483	while(strlen(buffer) == 1)
1484
1485	if (strlen(buffer) == 0) // this marks the end of the file and is our only way out
1486	break
1487	endif
1488
1489	sscanf buffer,"%x",dataval
1490
1491
1492	// type = (dataval & ~(2^32 - 2^30 -1))/2^30
1493
1494	// two most sig bits (31-30)
1495	type = (dataval & 0xC0000000)/1073741824 //right shift by 2^30
1496
1497	//
1498	// The defintions of the event types
1499	//
1500	//Constant ATXY = 0
1501	//Constant ATXYM = 2
1502	//Constant ATMIR = 1
1503	//Constant ATMAR = 3
1504	//
1505
1506	if(verbose > 0)
1507	verbose -= 1
1508	endif
1509	//
1510	switch(type)
1511	case ATXY: // 0
1512	if(verbose)
1513	printf "XY : "
1514	endif
1515
1516	// if the datavalue is == 0, just skip it now (it can only be interpreted as type 0, obviously)
1517	if(dataval == 0 && RemoveBadEvents == 1)
1518	numRemoved += 1
1519	//Print "zero at ii= ",ii
1520	break //don't increment ii
1521	endif
1522
1523	// if it's a pileup event, skip it now (this can be either type 0 or 2)
1524	bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1525	if(bit29 == 1 && RemoveBadEvents == 1)
1526	PPTime[tmpPP] = timeval
1527	PPEventNum[tmpPP] = ii
1528	tmpPP += 1
1529	numRemoved += 1
1530	break //don't increment ii
1531	endif
1532
1533	// xval = ~(dataval & ~(2^32 - 2^8)) & 127
1534	// yval = ((dataval & ~(2^32 - 2^16 ))/2^8) & 127
1535	// time_lsw = (dataval & ~(2^32 - 2^29))/2^16
1536
1537	xval = xMax - (dataval & 255) //last 8 bits (7-0)
1538	yval = (dataval & 65280)/256 //bits 15-8, right shift by 2^8
1539	time_lsw = (dataval & 536805376)/65536 //13 bits, 28-16, right shift by 2^16
1540
1541	timeval = trunc( nRollroll_time + (time_msw (8192)) + time_lsw ) //left shift msw by 2^13, then add in lsw, as an integer
1542	if (timeval > t_longest)
1543	t_longest = timeval
1544	endif
1545
1546
1547	// catch the "bad" events:
1548	// if an XY event follows a rollover, time_msw is 0 by definition, but does not immediately get
1549	// re-evalulated here. Throw out only the immediately following points where msw is still 8191
1550	if(rolloverHappened && RemoveBadEvents == 1)
1551	// maybe a bad event
1552	if(time_msw == 8191)
1553	badTimePt[numBad] = timeVal
1554	badEventNum[numBad] = ii
1555	numBad +=1
1556	numRemoved += 1
1557	else
1558	// time_msw has been reset, points are good now, so keep this one
1559	//// if(keep==decFac)
1560	xLoc[ii] = xval
1561	yLoc[ii] = yval
1562	timePt[ii] = timeval
1563
1564	// if(xval == 127 && yval == 0)
1565	// // check bit 29
1566	// bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1567	// Print "XY=127,0 : bit29 = ",bit29
1568	// endif
1569
1570	ii+=1
1571	rolloverHappened = 0
1572	//// keep = 0
1573	//// else
1574	//// keep += 1
1575	//// endif
1576	endif
1577	else
1578	// normal processing of good point, keep it
1579	//// if(keep==decFac)
1580	xLoc[ii] = xval
1581	yLoc[ii] = yval
1582	timePt[ii] = timeval
1583
1584	// if(xval == 127 && yval == 0)
1585	// // check bit 29
1586	// bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1587	// Printf "XY=127,0 : bit29 = %u : d=%u\r",bit29,dataval
1588	// endif
1589	ii+=1
1590	//// keep = 0
1591	//// else
1592	//// keep += 1
1593	//// endif
1594	endif
1595
1596
1597	if(verbose)
1598	// printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1599	printf "d=%u : t=%u : msw=%u : lsw=%u : %u : %u \r",dataval,timeval,time_msw,time_lsw,xval,yval
1600	endif
1601
1602	// verbose = 0
1603	break
1604	case ATXYM: // 2
1605	if(verbose)
1606	printf "XYM : "
1607	endif
1608
1609	// if it's a pileup event, skip it now (this can be either type 0 or 2)
1610	// - but can I do this if this is an XY-time event? This will lead to a wrong time, and a time
1611	// assigned to an XY (0,0)...
1612	// bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1613	// if(bit29 == 1 && RemoveBadEvents == 1)
1614	// Print "*****Bit 29 (PP) event set for Type==2, but not handled, ii = ",ii
1615	//// break //don't increment ii
1616	// endif
1617
1618	// xval = ~(dataval & ~(2^32 - 2^8)) & 127
1619	// yval = ((dataval & ~(2^32 - 2^16 ))/2^8) & 127
1620	// time_lsw = (dataval & ~(2^32 - 2^29 ))/2^16 //this method gives a FP result!! likely since the "^" operation gives FP result...
1621
1622	xval = xMax - (dataval & 255) //last 8 bits (7-0)
1623	yval = (dataval & 65280)/256 //bits 15-8, right shift by 2^8
1624
1625	time_lsw = (dataval & 536805376)/65536 //13 bits, 28-16, right shift by 2^16 (result is integer)
1626
1627	if(verbose)
1628	// printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1629	printf "%u : %u : %u : %u\r",dataval,timeval,xval,yval
1630	endif
1631
1632	//// if(keep==decFac) //don't reset keep yet, do this only when ii increments
1633	xLoc[ii] = xval
1634	yLoc[ii] = yval
1635	//// endif
1636
1637	// don't fill in the time yet, or increment the index ii
1638	// the next event MUST be ATMIR with the MSW time bits
1639	//
1640	// verbose = 0
1641	break
1642	case ATMIR: // 1
1643	if(verbose)
1644	printf "MIR : "
1645	endif
1646
1647	time_msw = (dataval & 536805376)/65536 //13 bits, 28-16, right shift by 2^16
1648	timeval = trunc( nRollroll_time + (time_msw (8192)) + time_lsw )
1649	if (timeval > t_longest)
1650	t_longest = timeval
1651	endif
1652	if(verbose)
1653	// printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1654	printf "d=%u : t=%u : msw=%u : lsw=%u : tlong=%u\r",dataval,timeval,time_msw,time_lsw,t_longest
1655	endif
1656
1657	// the XY position was in the previous event ATXYM
1658	//// if(keep == decFac)
1659	timePt[ii] = timeval
1660	//// endif
1661
1662	bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1663	if(bit29 != 0) // bit 29 set is a T0 event
1664	//Printf "bit29 = 1 at ii = %d : type = %d\r",ii,type
1665	T0Time[tmpT0] = timeval
1666	T0EventNum[tmpT0] = ii
1667	tmpT0 += 1
1668	// reset nRoll = 0 for calcluating the time
1669	nRoll = 0
1670	endif
1671
1672	//// if(keep == decFac)
1673	ii+=1
1674	//// keep = 0
1675	//// endif
1676	// verbose = 0
1677	break
1678	case ATMAR: // 3
1679	if(verbose3)
1680	// verbose = 15
1681	// verbose = 2
1682	printf "MAR : "
1683	endif
1684
1685	// do something with the rollover event?
1686
1687	// check bit 29
1688	bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1689	nRoll += 1
1690	// not doing anything with these bits yet
1691	bit28 = (dataval & 0x10000000)/268435456 //bit 28 only, shift by 2^28
1692	bit27 = (dataval & 0x08000000)/134217728 //bit 27 only, shift by 2^27
1693
1694	if(verbose3)
1695	printf "d=%u : b29=%u : b28=%u : b27=%u : #Roll=%u \r",dataval,bit29, bit28, bit27,nRoll
1696	endif
1697
1698	if(bit29 != 0) // bit 29 set is a T0 event
1699	//Printf "bit29 = 1 at ii = %d : type = %d\r",ii,type
1700	T0Time[tmpT0] = timeval
1701	T0EventNum[tmpT0] = ii
1702	tmpT0 += 1
1703	// reset nRoll = 0 for calcluating the time
1704	nRoll = 0
1705	endif
1706
1707	rolloverHappened = 1
1708
1709	break
1710	endswitch
1711
1712	// if(ii<18)
1713	// printf "TYPE=%d : ii=%d : d=%u : t=%u : msw=%u : lsw=%u : %u : %u \r",type,ii,dataval,timeval,time_msw,time_lsw,xval,yval
1714	// endif
1715
1716	while(1)
1717
1718	Close fileref
1719
1720	printf("Igor full file read done in ")
1721	toc()
1722
1723	Print "Events removed (Igor) = ",numRemoved
1724
1725	sPrintf tmpStr,"\rBad Rollover Events = %d (%4.4g %% of events)",numBad,numBad/numXYevents*100
1726	dispStr += tmpStr
1727	sPrintf tmpStr,"\rTotal Events Removed = %d (%4.4g %% of events)",numRemoved,numRemoved/numXYevents*100
1728	dispStr += tmpStr
1729	SetDataFolder root:
1730
1731	return(0)
1732
1733	End
1734
1735	//////////////
1736	//
1737	// This calls the XOP, as an operation to load the events
1738	//
1739	// -- it's about 35x faster than the Igor code, so I guess that's OK.
1740	//
1741	// conditional compile the whole inner workings in case XOP is not present
1742	Function LoadEvents_XOP()
1743	#if (exists("EventLoadWave")==4)
1744
1745	// NVAR time_msw = root:Packages:NIST:Event:gEvent_time_msw
1746	// NVAR time_lsw = root:Packages:NIST:Event:gEvent_time_lsw
1747	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
1748
1749	SVAR filepathstr = root:Packages:NIST:Event:gEvent_logfile
1750	SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
1751
1752	SetDataFolder root:Packages:NIST:Event
1753
1754
1755
1756	Variable fileref
1757	String buffer
1758	String fileStr,tmpStr
1759	Variable dataval,timeval,type,numLines,verbose,verbose3
1760	Variable xval,yval,rollBit,nRoll,roll_time,bit29,bit28,bit27
1761	Variable ii,flaggedEvent,rolloverHappened,numBad=0,tmpPP=0,tmpT0=0
1762	Variable Xmax, yMax
1763
1764	xMax = 127 // number the detector from 0->127
1765	yMax = 127
1766
1767	numLines = 0
1768
1769	//Have to declare local variables for Loadwave so that this compiles without XOP.
1770	String S_waveNames
1771	// and those for the XOP
1772	Variable V_nXYevents,V_num1,V_num2,V_num3,V_num0,V_totBytes,V_numPP,V_numT0,V_numDL,V_numFF,V_numZero
1773	Variable V_numBad,V_numRemoved
1774
1775	// what I really need is the number of XY events
1776	Variable numXYevents,num1,num2,num3,num0,totBytes,numPP,numT0,numDL,numFF,numZero
1777	Variable numRemoved
1778	numXYevents = 0
1779	num0 = 0
1780	num1 = 0
1781	num2 = 0
1782	num3 = 0
1783	numPP = 0
1784	numT0 = 0
1785	numDL = 0
1786	numFF = 0
1787	numZero = 0
1788	numRemoved = 0
1789
1790	// get the total number of bytes in the file
1791	Open/R fileref as filepathstr
1792	FStatus fileref
1793	Close fileref
1794
1795	totBytes = V_logEOF
1796	Print "total bytes = ", totBytes
1797
1798	if(EventDataType(filePathStr) != 1) //if not hst extension of raw data
1799	Abort "Data must be raw event data with file extension .hst for this operation"
1800	return(0)
1801	endif
1802	//
1803	// Print "scan only"
1804	// tic()
1805	// EventLoadWave/R/N=EventWave/W filepathstr
1806	// toc()
1807
1808	////
1809	//
1810	// use the XOP operation to load in the data
1811	// -- this does everything - the pre-scan and creating the waves
1812	//
1813	// need to zero the waves before loading, just in case
1814	//
1815
1816	NVAR removeBadEvents = root:Packages:NIST:Event:gRemoveBadEvents
1817
1818	tic()
1819
1820	// Wave/Z wave0=wave0
1821	// Wave/Z wave1=wave1
1822	// Wave/Z wave2=wave2
1823	//
1824	// if(WaveExists(wave0))
1825	// MultiThread wave0=0
1826	// endif
1827	// if(WaveExists(wave1))
1828	// MultiThread wave1=0
1829	// endif
1830	// if(WaveExists(wave2))
1831	// MultiThread wave2=0
1832	// endif
1833
1834	if(removeBadEvents)
1835	EventLoadWave/R/N=EventWave filepathstr
1836	else
1837	EventLoadWave/N=EventWave filepathstr
1838	endif
1839
1840
1841	Print "XOP files loaded = ",S_waveNames
1842
1843	//// -- copy the waves over to xLoc,yLoc,timePt
1844	Wave/Z EventWave0=EventWave0
1845	Wave/Z EventWave1=EventWave1
1846	Wave/Z EventWave2=EventWave2
1847
1848
1849	Duplicate/O EventWave0,xLoc
1850	KillWaves/Z EventWave0
1851
1852	Duplicate/O EventWave1,yLoc
1853	KillWaves/Z EventWave1
1854
1855	Duplicate/O EventWave2,timePt
1856	KillWaves/Z EventWave2
1857
1858	// could do this, but rescaled time will neeed to be converted to SP (or DP)
1859	// and Igor loader was written with Make generating SP/DP waves
1860	// /I/U is unsigned 32-bit integer (for the time)
1861	// /B/U is unsigned 8-bit integer (max val=255) for the x and y values
1862
1863	// Redimension/B/U xLoc,yLoc
1864	// Redimension/I/U timePt
1865
1866	// access the variables from the XOP
1867	numT0 = V_numT0
1868	numPP = V_numPP
1869	num0 = V_num0
1870	num1 = V_num1
1871	num2 = V_num2
1872	num3 = V_num3
1873	numXYevents = V_nXYevents
1874	numZero = V_numZero
1875	numBad = V_numBad
1876	numRemoved = V_numRemoved
1877
1878	Print "(XOP) numT0 = ",numT0
1879	Print "num0 = ",num0
1880	Print "num1 = ",num1
1881	Print "num2 = ",num2
1882	Print "num3 = ",num3
1883
1884
1885	// dispStr will be displayed on the panel
1886	fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
1887
1888	sprintf tmpStr, "%s: %d total bytes\r",fileStr,totBytes
1889	dispStr = tmpStr
1890	sprintf tmpStr,"numXYevents = %d\r",numXYevents
1891	dispStr += tmpStr
1892	sprintf tmpStr,"PP = %d : ",numPP
1893	dispStr += tmpStr
1894	sprintf tmpStr,"ZeroData = %d\r",numZero
1895	dispStr += tmpStr
1896	sprintf tmpStr,"Rollover = %d",num3
1897	dispStr += tmpStr
1898
1899	toc()
1900
1901	Print "Events removed (XOP) = ",numRemoved
1902
1903	sPrintf tmpStr,"\rBad Rollover Events = %d (%4.4g %% of events)",numBad,numBad/numXYevents*100
1904	dispStr += tmpStr
1905	sPrintf tmpStr,"\rTotal Events Removed = %d (%4.4g %% of events)",numRemoved,numRemoved/numXYevents*100
1906	dispStr += tmpStr
1907
1908
1909	// simply to compile a table of # XY vs # bytes
1910	// Wave/Z nxy = root:numberXY
1911	// Wave/Z nBytes = root:numberBytes
1912	// if(WaveExists(nxy) && WaveExists(nBytes))
1913	// InsertPoints 0, 1, nxy,nBytes
1914	// nxy[0] = numXYevents
1915	// nBytes[0] = totBytes
1916	// endif
1917
1918	SetDataFolder root:
1919
1920	#endif
1921	return(0)
1922
1923	End
1924
1925	//////////////
1926
1927	Proc BinEventBarGraph()
1928
1929	DoWindow/F EventBarGraph
1930	if(V_flag == 0)
1931	PauseUpdate; Silent 1 // building window...
1932	String fldrSav0= GetDataFolder(1)
1933	SetDataFolder root:Packages:NIST:Event:
1934	Display /W=(110,705,610,1132)/N=EventBarGraph /K=1 binCount vs binEndTime
1935	SetDataFolder fldrSav0
1936	ModifyGraph mode=5
1937	ModifyGraph marker=19
1938	ModifyGraph lSize=2
1939	ModifyGraph rgb=(0,0,0)
1940	ModifyGraph msize=2
1941	ModifyGraph hbFill=2
1942	ModifyGraph gaps=0
1943	ModifyGraph usePlusRGB=1
1944	ModifyGraph toMode=1
1945	ModifyGraph useBarStrokeRGB=1
1946	ModifyGraph standoff=0
1947	SetAxis left 0,*
1948	Label bottom "\\Z14Time (seconds)"
1949	Label left "\\Z14Number of Events"
1950	endif
1951	End
1952
1953
1954	Proc ShowBinTable()
1955
1956	DoWindow/F BinEventTable
1957	if(V_flag == 0)
1958	PauseUpdate; Silent 1 // building window...
1959	String fldrSav0= GetDataFolder(1)
1960	SetDataFolder root:Packages:NIST:Event:
1961	Edit/W=(498,699,1003,955) /K=1/N=BinEventTable binCount,binEndTime,timeWidth
1962	ModifyTable format(Point)=1,sigDigits(binEndTime)=8,width(binEndTime)=100
1963	SetDataFolder fldrSav0
1964	endif
1965	EndMacro
1966
1967
1968	// only show the first 1500 data points
1969	//
1970	Proc ShowRescaledTimeGraph()
1971
1972	DoWindow/F RescaledTimeGraph
1973	if(V_flag == 0)
1974	PauseUpdate; Silent 1 // building window...
1975	String fldrSav0= GetDataFolder(1)
1976	SetDataFolder root:Packages:NIST:Event:
1977	Display /W=(25,44,486,356)/K=1/N=RescaledTimeGraph rescaledTime
1978	SetDataFolder fldrSav0
1979	ModifyGraph mode=0
1980	ModifyGraph marker=19
1981	ModifyGraph rgb(rescaledTime)=(0,0,0)
1982	ModifyGraph msize=1
1983	// SetAxis/A=2 left //only autoscale the visible data (based on the bottom limits)
1984	SetAxis bottom 0,1500
1985	ErrorBars rescaledTime OFF
1986	Label left "\\Z14Time (seconds)"
1987	Label bottom "\\Z14Event number"
1988	ShowInfo
1989	endif
1990
1991	EndMacro
1992
1993
1994
1995	Proc ExportSlicesAsVAX(firstNum,prefix)
1996	Variable firstNum=1
1997	String prefix="SAMPL"
1998
1999	SaveSlicesAsVAX(firstNum,prefix[0,4]) //make sure that the prefix is 5 chars
2000	End
2001
2002	//////// procedures to be able to export the slices as RAW VAX files.
2003	//
2004	// 1- load the raw data file to use the header (it must already be in RAW)
2005	// 1.5- copy the raw data to the temp folder (STO)
2006	// 1.7- ask for the prefix and starting run number (these are passed in)
2007	// 2- copy the slice of data to the temp folder (STO)
2008	// 3- touch up the time/counts in the slice header values in STO
2009	// 4- write out the VAX file
2010	// 5- repeat (2-4) for the number of slices
2011	//
2012	//
2013	Function SaveSlicesAsVAX(firstNum,prefix)
2014	Variable firstNum
2015	String prefix
2016
2017	DoAlert 1,"Is the full data file loaded as a RAW data file? If not, load it and start over..."
2018	if(V_flag == 2)
2019	return (0)
2020	endif
2021
2022	// copy the contents of RAW to STO so I can work from there
2023	CopyWorkContents("RAW","STO")
2024
2025	// now declare all of the waves, now that they are sure to be there
2026
2027	WAVE slicedData=root:Packages:NIST:Event:slicedData
2028	Make/O/D/N=(128,128) curSlice
2029
2030	NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
2031	WAVE binEndTime = root:Packages:NIST:Event:binEndTime
2032
2033	Wave rw=root:Packages:NIST:STO:realsRead
2034	Wave iw=root:Packages:NIST:STO:integersRead
2035	Wave/T tw=root:Packages:NIST:STO:textRead
2036	Wave data=root:Packages:NIST:STO:data
2037	Wave linear_data=root:Packages:NIST:STO:linear_data
2038
2039
2040	Wave rw_raw=root:Packages:NIST:RAW:realsRead
2041	Wave iw_raw=root:Packages:NIST:RAW:integersRead
2042	Wave/T tw_raw=root:Packages:NIST:RAW:textRead
2043
2044	// for generating the alphanumeric
2045	String timeStr= secs2date(datetime,-1)
2046	String monthStr=StringFromList(1, timeStr ,"/")
2047	String numStr="",labelStr
2048
2049	Variable ii,err,binFraction
2050
2051	for(ii=0;ii<nslices;ii+=1)
2052
2053	//get the current slice and put it in the STO folder
2054	curSlice = slicedData[p][q][ii]
2055	data = curSlice
2056	linear_data = curSlice
2057
2058	// touch up the header as needed
2059	// count time = iw[2]
2060	// monCt = rw[0]
2061	// detCt = rw[2]
2062	//tw[0] must now be the file name
2063	//
2064	// count time = fraction of total binning * total count time
2065	binFraction = (binEndTime[ii+1]-binEndTime[ii])/(binEndTime[nslices]-binEndTime[0])
2066
2067	iw[2] = trunc(binFraction*iw_raw[2])
2068	rw[0] = trunc(binFraction*rw_raw[0])
2069	rw[2] = sum(curSlice,-inf,inf) //total counts in slice
2070
2071	if(firstNum<10)
2072	numStr = "00"+num2str(firstNum)
2073	else
2074	if(firstNum<100)
2075	numStr = "0"+num2str(firstNum)
2076	else
2077	numStr = num2str(firstNum)
2078	Endif
2079	Endif
2080	tw[0] = prefix+numstr+".SA2_EVE_"+(num2char(str2num(monthStr)+64))+numStr
2081	labelStr = tw_raw[6]
2082
2083	labelStr = PadString(labelStr,60,0x20) //60 fortran-style spaces
2084	tw[6] = labelStr[0,59]
2085
2086	//write out the file - this uses the tw[0] and home path
2087	Write_VAXRaw_Data("STO","",0)
2088
2089	//increment the run number, alpha
2090	firstNum += 1
2091	endfor
2092
2093	return(0)
2094	End
2095
2096
2097
2098
2099
2100	/////////////
2101	//The histogramming
2102	//
2103	// 6 AUG 2012
2104	//
2105	// from Igor Exchange, RGerkin
2106	// http://www.igorexchange.com/node/1373
2107	// -- see the related thread on the mailing list
2108	//
2109	//Function Setup_JointHistogram()
2110	//
2111	//// tic()
2112	//
2113	// make/D /o/n=1000000 data1=gnoise(1), data2=gnoise(1)
2114	// make/D /o/n=(25,25) myHist
2115	// setscale x,-3,3,myHist
2116	// setscale y,-3,3,myHist
2117	// IndexForHistogram(data1,data2,myhist)
2118	// Wave index=SavedIndex
2119	// EV_JointHistogram(data1,data2,myHist,index)
2120	// NewImage myHist
2121	//
2122	//// toc()
2123	//
2124	//End
2125
2126
2127	Function EV_JointHistogram(w0,w1,hist,index)
2128	wave w0,w1,hist,index
2129
2130	variable bins0=dimsize(hist,0)
2131	variable bins1=dimsize(hist,1)
2132	variable n=numpnts(w0)
2133	variable left0=dimoffset(hist,0)
2134	variable left1=dimoffset(hist,1)
2135	variable right0=left0+bins0*dimdelta(hist,0)
2136	variable right1=left1+bins1*dimdelta(hist,1)
2137
2138	// Compute the histogram and redimension it.
2139	histogram /b={0,1,bins0*bins1} index,hist
2140	redimension/D /n=(bins0,bins1) hist // Redimension to 2D.
2141	setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension.
2142	setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension.
2143	End
2144
2145
2146	// histogram with a point range
2147	//
2148	// x- just need to send x2pnt or findLevel, or something similar to define the POINT
2149	// values
2150	//
2151	// x- can also speed this up since the index only needs to be done once, so the
2152	// histogram operation can be done separately, as the bins require
2153	//
2154	//
2155	Function JointHistogramWithRange(w0,w1,hist,index,pt1,pt2)
2156	wave w0,w1,hist,index
2157	Variable pt1,pt2
2158
2159	variable bins0=dimsize(hist,0)
2160	variable bins1=dimsize(hist,1)
2161	variable n=numpnts(w0)
2162	variable left0=dimoffset(hist,0)
2163	variable left1=dimoffset(hist,1)
2164	variable right0=left0+bins0*dimdelta(hist,0)
2165	variable right1=left1+bins1*dimdelta(hist,1)
2166
2167	// Compute the histogram and redimension it.
2168	histogram /b={0,1,bins0*bins1}/R=[pt1,pt2] index,hist
2169	redimension/D /n=(bins0,bins1) hist // Redimension to 2D.
2170	setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension.
2171	setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension.
2172	End
2173
2174
2175	// just does the indexing, creates wave SavedIndex in the current folder for the index
2176	//
2177	Function IndexForHistogram(w0,w1,hist)
2178	wave w0,w1,hist
2179
2180	variable bins0=dimsize(hist,0)
2181	variable bins1=dimsize(hist,1)
2182	variable n=numpnts(w0)
2183	variable left0=dimoffset(hist,0)
2184	variable left1=dimoffset(hist,1)
2185	variable right0=left0+bins0*dimdelta(hist,0)
2186	variable right1=left1+bins1*dimdelta(hist,1)
2187
2188	// Scale between 0 and the number of bins to create an index wave.
2189	if(ThreadProcessorCount<4) // For older machines, matrixop is faster.
2190	matrixop /free idx=round(bins0(w0-left0)/(right0-left0))+bins0round(bins1*(w1-left1)/(right1-left1))
2191	else // For newer machines with many cores, multithreading with make is faster.
2192	make/free/n=(n) idx
2193	multithread idx=round(bins0(w0-left0)/(right0-left0))+bins0round(bins1*(w1-left1)/(right1-left1))
2194	endif
2195
2196	KillWaves/Z SavedIndex
2197	MoveWave idx,SavedIndex
2198
2199	// // Compute the histogram and redimension it.
2200	// histogram /b={0,1,bins0*bins1} idx,hist
2201	// redimension /n=(bins0,bins1) hist // Redimension to 2D.
2202	// setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension.
2203	// setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension.
2204	End
2205
2206
2207
2208
2209
2210	////////////// Post-processing of the event mode data
2211	Proc ShowEventCorrectionPanel()
2212	DoWindow/F EventCorrectionPanel
2213	if(V_flag ==0)
2214	EventCorrectionPanel()
2215	EndIf
2216	End
2217
2218	Proc EventCorrectionPanel()
2219
2220	PauseUpdate; Silent 1 // building window...
2221	SetDataFolder root:Packages:NIST:Event:
2222
2223	if(exists("rescaledTime") == 1)
2224	Display /W=(35,44,761,533)/K=2 rescaledTime
2225	DoWindow/C EventCorrectionPanel
2226	ModifyGraph mode=0
2227	ModifyGraph marker=19
2228	ModifyGraph rgb=(0,0,0)
2229	ModifyGraph msize=1
2230	ErrorBars rescaledTime OFF
2231	Label left "\\Z14Time (seconds)"
2232	Label bottom "\\Z14Event number"
2233	SetAxis bottom 0,0.10*numpnts(rescaledTime) //show 1st 10% of data for speed in displaying
2234
2235	ControlBar 100
2236	Button button0,pos={18,12},size={70,20},proc=EC_AddCursorButtonProc,title="Cursors"
2237	Button button1,pos={153,12},size={80,20},proc=EC_AddTimeButtonProc,title="Add time"
2238	Button button2,pos={153,38},size={80,20},proc=EC_SubtractTimeButtonProc,title="Subtr time"
2239	Button button3,pos={153,64},size={90,20},proc=EC_TrimPointsButtonProc,title="Trim points"
2240	Button button4,pos={295+150,12},size={90,20},proc=EC_SaveWavesButtonProc,title="Save Waves"
2241	Button button5,pos={285,64},size={100,20},proc=EC_FindOutlierButton,title="Find Outlier"
2242	Button button6,pos={18,38},size={80,20},proc=EC_ShowAllButtonProc,title="All Data"
2243	Button button7,pos={683,12},size={30,20},proc=EC_HelpButtonProc,title="?"
2244	Button button8,pos={658,72},size={60,20},proc=EC_DoneButtonProc,title="Done"
2245
2246	Button button9,pos={285,12},size={110,20},proc=EC_FindStepButton_down,title="Find Step Down"
2247	Button button10,pos={285,38},size={110,20},proc=EC_FindStepButton_up,title="Find Step Up"
2248	Button button11,pos={295+150,38},size={110,20},proc=EC_DoDifferential,title="Differential"
2249	Button button12,pos={295+150,64},size={110,20},proc=EC_AddFindNext,title="Add Find Next"
2250	Button button13,pos={285+120,12},size={20,20},proc=EC_NudgeCursor,title=">"
2251
2252	else
2253	DoAlert 0, "Please load some event data, then you'll have something to edit."
2254	endif
2255
2256	SetDataFolder root:
2257
2258	EndMacro
2259
2260
2261	Function EC_NudgeCursor(ba)
2262	STRUCT WMButtonAction &ba
2263
2264	switch( ba.eventCode )
2265	case 2: // mouse up
2266	// click code here
2267	// nudge the leftmost cursor to the right two points
2268	// presumably to bypass a desired time step
2269	//
2270
2271	Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
2272	Variable ptA,ptB,lo
2273	ptA = pcsr(A)
2274	ptB = pcsr(B)
2275	lo=min(ptA,ptB)
2276
2277	if(lo == ptA)
2278	Cursor/P A rescaledTime ptA+2 //at the point + 2
2279	Print "Nudged cursor A two points right"
2280	else
2281	Cursor/P B rescaledTime ptB+2 //at the point + 2
2282	Print "Nudged cursor B two points right"
2283	endif
2284
2285	break
2286	case -1: // control being killed
2287	break
2288	endswitch
2289
2290	return(0)
2291	End
2292
2293	Function EC_AddFindNext(ba)
2294	STRUCT WMButtonAction &ba
2295
2296	switch( ba.eventCode )
2297	case 2: // mouse up
2298	// click code here
2299	// add time
2300	EC_AddTimeButtonProc(ba)
2301
2302	// re-do the differential
2303	EC_DoDifferential("")
2304
2305	// find the next step down
2306	EC_FindStepButton_down("")
2307
2308	break
2309	case -1: // control being killed
2310	break
2311	endswitch
2312
2313	return(0)
2314	End
2315
2316
2317
2318	Function EC_AddCursorButtonProc(ba) : ButtonControl
2319	STRUCT WMButtonAction &ba
2320
2321	switch( ba.eventCode )
2322	case 2: // mouse up
2323	// click code here
2324	SetDataFolder root:Packages:NIST:Event:
2325
2326	Wave rescaledTime = rescaledTime
2327	Cursor/P A rescaledTime 0
2328	Cursor/P B rescaledTime numpnts(rescaledTime)-1
2329	ShowInfo
2330	SetDataFolder root:
2331	break
2332	case -1: // control being killed
2333	break
2334	endswitch
2335
2336	return 0
2337	End
2338
2339	// updates the longest time (as does every operation of adjusting the data)
2340	//
2341	Function EC_AddTimeButtonProc(ba) : ButtonControl
2342	STRUCT WMButtonAction &ba
2343
2344	switch( ba.eventCode )
2345	case 2: // mouse up
2346	// click code here
2347	SetDataFolder root:Packages:NIST:Event:
2348
2349	Wave rescaledTime = rescaledTime
2350	Wave timePt = timePt
2351	Variable rollTime,rollTicks,ptA,ptB,lo,hi
2352
2353	rollTicks = 2^26 // in ticks
2354	rollTime = 2^26*1e-7 // in seconds
2355	ptA = pcsr(A)
2356	ptB = pcsr(B)
2357	lo=min(ptA,ptB)
2358	hi=max(ptA,ptB)
2359
2360	MultiThread timePt[lo,hi] += rollTicks
2361	MultiThread rescaledTime[lo,hi] += rollTime
2362
2363	// updates the longest time (as does every operation of adjusting the data)
2364	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2365	t_longest = waveMax(rescaledTime)
2366
2367	SetDataFolder root:
2368	break
2369	case -1: // control being killed
2370	break
2371	endswitch
2372
2373	return 0
2374	End
2375
2376	Function EC_SubtractTimeButtonProc(ba) : ButtonControl
2377	STRUCT WMButtonAction &ba
2378
2379	switch( ba.eventCode )
2380	case 2: // mouse up
2381	// click code here
2382	SetDataFolder root:Packages:NIST:Event:
2383
2384	Wave rescaledTime = rescaledTime
2385	Wave timePt = timePt
2386	Variable rollTime,rollTicks,ptA,ptB,lo,hi
2387
2388	rollTicks = 2^26 // in ticks
2389	rollTime = 2^26*1e-7 // in seconds
2390	ptA = pcsr(A)
2391	ptB = pcsr(B)
2392	lo=min(ptA,ptB)
2393	hi=max(ptA,ptB)
2394
2395	MultiThread timePt[lo,hi] -= rollTicks
2396	MultiThread rescaledTime[lo,hi] -= rollTime
2397
2398	// updates the longest time (as does every operation of adjusting the data)
2399	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2400	t_longest = waveMax(rescaledTime)
2401
2402	SetDataFolder root:
2403
2404	break
2405	case -1: // control being killed
2406	break
2407	endswitch
2408
2409	return 0
2410	End
2411
2412	// points removed are inclusive
2413	//
2414	// put both cursors on the same point to remove just that single point
2415	//
2416	Function EC_TrimPointsButtonProc(ba) : ButtonControl
2417	STRUCT WMButtonAction &ba
2418
2419	switch( ba.eventCode )
2420	case 2: // mouse up
2421	// click code here
2422	SetDataFolder root:Packages:NIST:Event:
2423
2424	Wave rescaledTime = rescaledTime
2425	Wave timePt = timePt
2426	Wave xLoc = xLoc
2427	Wave yLoc = yLoc
2428	Variable rollTime,ptA,ptB,numElements,lo,hi
2429
2430	rollTime = 2^26*1e-7 // in seconds
2431	ptA = pcsr(A)
2432	ptB = pcsr(B)
2433	lo=min(ptA,ptB)
2434	hi=max(ptA,ptB)
2435	numElements = abs(ptA-ptB)+1 //so points removed are inclusive
2436	DeletePoints lo, numElements, rescaledTime,timePt,xLoc,yLoc
2437
2438	printf "Points %g to %g have been deleted in rescaledTime, timePt, xLoc, and yLoc\r",ptA,ptB
2439
2440	// updates the longest time (as does every operation of adjusting the data)
2441	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2442	t_longest = waveMax(rescaledTime)
2443
2444	SetDataFolder root:
2445
2446	break
2447	case -1: // control being killed
2448	break
2449	endswitch
2450
2451	return 0
2452	End
2453
2454	// un-sort the data first, then save it
2455	Function EC_SaveWavesButtonProc(ba) : ButtonControl
2456	STRUCT WMButtonAction &ba
2457
2458	switch( ba.eventCode )
2459	case 2: // mouse up
2460	// click code here
2461
2462	// Execute "UndoTheSorting()"
2463
2464	SetDataFolder root:Packages:NIST:Event:
2465
2466	Wave rescaledTime = rescaledTime
2467	Wave timePt = timePt
2468	Wave xLoc = xLoc
2469	Wave yLoc = yLoc
2470	Save/T xLoc,yLoc,timePt,rescaledTime //will ask for a name
2471
2472	SetDataFolder root:
2473	break
2474	case -1: // control being killed
2475	break
2476	endswitch
2477
2478	return 0
2479	End
2480
2481	//
2482	// this duplicates all of the bits that would be done if the "load" button was pressed
2483	//
2484	Function EC_ImportWavesButtonProc(ba) : ButtonControl
2485	STRUCT WMButtonAction &ba
2486
2487	switch( ba.eventCode )
2488	case 2: // mouse up
2489	// click code here
2490	SetDataFolder root:Packages:NIST:Event:
2491
2492	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2493	SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
2494	String tmpStr="",fileStr,filePathStr
2495
2496	// load in the waves, saved as Igor text to preserve the data type
2497	PathInfo/S catPathName //should set the next dialog to the proper path...
2498	filePathStr = PromptForPath("Pick the edited event data")
2499	if(EventDataType(filePathStr) != 2) //if not itx extension of edited data
2500	Abort "Data must be edited data with file extension .itx for this operation"
2501	return(0)
2502	endif
2503	LoadWave/T/O/P=catPathName filePathStr
2504	// filePathStr = S_fileName
2505
2506	if(strlen(S_fileName) == 0)
2507	//user cancelled
2508	DoAlert 0,"No file selected, nothing done."
2509	return(0)
2510	endif
2511
2512	NVAR mode = root:Packages:NIST:Event:gEvent_Mode // ==0 for "stream", ==1 for Oscillatory
2513	// clear out the old sort index, if present, since new data is being loaded
2514	KillWaves/Z OscSortIndex
2515	Wave timePt=timePt
2516	Wave rescaledTime=rescaledTime
2517
2518	t_longest = waveMax(rescaledTime) //should be the last point
2519
2520
2521	fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
2522	sprintf tmpStr, "%s: a user-modified event file\r",fileStr
2523	dispStr = tmpStr
2524
2525	SetDataFolder root:
2526	break
2527	case -1: // control being killed
2528	break
2529	endswitch
2530
2531	return 0
2532	End
2533
2534
2535	Function EC_ShowAllButtonProc(ba) : ButtonControl
2536	STRUCT WMButtonAction &ba
2537
2538	switch( ba.eventCode )
2539	case 2: // mouse up
2540	// click code here
2541	SetAxis/A
2542	break
2543	case -1: // control being killed
2544	break
2545	endswitch
2546
2547	return 0
2548	End
2549
2550	Function EC_HelpButtonProc(ba) : ButtonControl
2551	STRUCT WMButtonAction &ba
2552
2553	switch( ba.eventCode )
2554	case 2: // mouse up
2555	// click code here
2556	DisplayHelpTopic/Z "Event Mode Data[Correcting for things that go wrong]"
2557	break
2558	case -1: // control being killed
2559	break
2560	endswitch
2561
2562	return 0
2563	End
2564
2565	Function EC_DoneButtonProc(ba) : ButtonControl
2566	STRUCT WMButtonAction &ba
2567
2568	switch( ba.eventCode )
2569	case 2: // mouse up
2570	// click code here
2571	DoWindow/K EventCorrectionPanel
2572	break
2573	case -1: // control being killed
2574	break
2575	endswitch
2576
2577	return 0
2578	End
2579
2580	//upDown 5 or -5 looks for spikes +5 or -5 std deviations from mean
2581	//
2582	// will search from the leftmost cursor to the end. this allows skipping of oscillations
2583	// that are not timing errors. It may introduce other issues, but we'll see what happens
2584	//
2585	Function PutCursorsAtStep(upDown)
2586	Variable upDown
2587
2588	SetDataFolder root:Packages:NIST:Event:
2589
2590	Wave rescaledTime=rescaledTime
2591	Wave rescaledTime_DIF=rescaledTime_DIF
2592	Variable avg,pt,zoom
2593	Variable ptA,ptB,startPt
2594
2595	zoom = 200 //points in each direction
2596
2597	WaveStats/M=1/Q rescaledTime_DIF
2598	avg = V_avg
2599
2600	ptA = pcsr(A)
2601	ptB = pcsr(B)
2602	startPt = min(ptA,ptB)
2603
2604	FindLevel/P/Q/R=[startPt] rescaledTime_DIF avg*upDown
2605	if(V_flag==0)
2606	pt = V_levelX
2607	WaveStats/Q/R=[pt-zoom,pt+zoom] rescaledTime // find the max/min y-values within the point range
2608	else
2609	Print "Level not found"
2610	// return(0)
2611	endif
2612
2613	Variable loLeft,hiLeft, loBottom,hiBottom
2614	loLeft = V_min*0.98 //+/- 2%
2615	hiLeft = V_max*1.02
2616
2617	SetAxis left loLeft,hiLeft
2618	SetAxis bottom pnt2x(rescaledTime,pt-zoom),pnt2x(rescaledTime,pt+zoom)
2619
2620	Cursor/P A rescaledTime pt+2 //at the point
2621	Cursor/P B rescaledTime numpnts(rescaledTime)-1 //at the end
2622
2623	SetDataFolder root:
2624
2625	return(0)
2626	End
2627
2628
2629	// find the max (or min) of the rescaled time set
2630	// and place both cursors there
2631	Function fFindOutlier()
2632
2633	SetDataFolder root:Packages:NIST:Event:
2634
2635	Wave rescaledTime=rescaledTime
2636	Variable avg,pt,zoom,maxPt,minPt,maxVal,minVal
2637
2638	zoom = 200 //points in each direction
2639
2640	WaveStats/M=1/Q rescaledTime
2641	maxPt = V_maxLoc
2642	minPt = V_minLoc
2643	avg = V_avg
2644	maxVal = abs(V_max)
2645	minVal = abs(V_min)
2646
2647	pt = abs(maxVal - avg) > abs(minVal - avg) ? maxPt : minPt
2648
2649	// Variable loLeft,hiLeft, loBottom,hiBottom
2650	// loLeft = V_min*0.98 //+/- 2%
2651	// hiLeft = V_max*1.02
2652
2653	// SetAxis left loLeft,hiLeft
2654	// SetAxis bottom pnt2x(rescaledTime,pt-zoom),pnt2x(rescaledTime,pt+zoom)
2655
2656	Cursor/P A rescaledTime pt //at the point
2657	Cursor/P B rescaledTime pt //at the same point
2658
2659	SetDataFolder root:
2660
2661	return(0)
2662	End
2663
2664	Function EC_FindStepButton_down(ctrlName) : ButtonControl
2665	String ctrlName
2666
2667	// Variable upDown = -5
2668	NVAR upDown = root:Packages:NIST:Event:gStepTolerance
2669
2670	PutCursorsAtStep(-1*upDown)
2671
2672	return(0)
2673	end
2674
2675
2676	Function EC_FindStepButton_up(ctrlName) : ButtonControl
2677	String ctrlName
2678
2679	// Variable upDown = 5
2680	NVAR upDown = root:Packages:NIST:Event:gStepTolerance
2681
2682	PutCursorsAtStep(upDown)
2683
2684	return(0)
2685	end
2686
2687	// if the Trim button section is uncommented, it's "Zap outlier"
2688	//
2689	Function EC_FindOutlierButton(ctrlName) : ButtonControl
2690	String ctrlName
2691
2692	fFindOutlier()
2693	//
2694	// STRUCT WMButtonAction ba
2695	// ba.eventCode = 2
2696	//
2697	// EC_TrimPointsButtonProc(ba)
2698
2699	return(0)
2700	end
2701
2702	Function EC_DoDifferential(ctrlName) : ButtonControl
2703	String ctrlName
2704
2705	DifferentiatedTime()
2706	DoWindow/F EventCorrectionPanel
2707
2708	//if trace is not on graph, add it
2709	SetDataFolder root:Packages:NIST:Event:
2710
2711	String list = WaveList("*_DIF", ";", "WIN:EventCorrectionPanel")
2712	if(strlen(list) == 0)
2713	AppendToGraph/R rescaledTime_DIF
2714	ModifyGraph msize=1,rgb(rescaledTime_DIF)=(65535,0,0)
2715	ReorderTraces rescaledTime,{rescaledTime_DIF} // put the differential behind the event data
2716	endif
2717	SetDataFolder root:
2718	return(0)
2719	end
2720
2721	////////////// Custom Bins /////////////////////
2722	//
2723	//
2724	//
2725	// make sure that the bins are defined and the waves exist before
2726	// trying to draw the panel
2727	//
2728	Proc Show_CustomBinPanel()
2729	DoWindow/F CustomBinPanel
2730	if(V_flag ==0)
2731	Init_CustomBins()
2732	CustomBinPanel()
2733	EndIf
2734	End
2735
2736
2737	Function Init_CustomBins()
2738
2739	NVAR nSlice = root:Packages:NIST:Event:gEvent_nslices
2740	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2741
2742	Variable/G root:Packages:NIST:Event:gEvent_ForceTmaxBin=1 //==1 to enforce t_longest in user-defined custom bins
2743
2744	SetDataFolder root:Packages:NIST:Event:
2745
2746	Make/O/D/N=(nSlice) timeWidth
2747	Make/O/D/N=(nSlice+1) binEndTime,binCount
2748
2749	timeWidth = t_longest/nslice
2750	binEndTime = p
2751	binCount = p+1
2752
2753	SetDataFolder root:
2754
2755	return(0)
2756	End
2757
2758	////////////////
2759	//
2760	// Allow custom definitions of the bin widths
2761	//
2762	// Define by the number of bins, and the time width of each bin
2763	//
2764	// This shares the number of slices and the maximum time with the main panel
2765	//
2766	Proc CustomBinPanel()
2767	PauseUpdate; Silent 1 // building window...
2768	NewPanel /W=(130,44,851,455)/K=2 /N=CustomBinPanel
2769	DoWindow/C CustomBinPanel
2770	ModifyPanel fixedSize=1//,noEdit =1
2771	SetDrawLayer UserBack
2772
2773	Button button0,pos={654,42}, size={50,20},title="Done",fSize=12
2774	Button button0,proc=CB_Done_Proc
2775	Button button1,pos={663,14},size={40,20},proc=CB_HelpButtonProc,title="?"
2776	Button button2,pos={216,42},size={80,20},title="Update",proc=CB_UpdateWavesButton
2777	SetVariable setvar1,pos={23,13},size={160,20},title="Number of slices",fSize=12
2778	SetVariable setvar1,proc=CB_NumSlicesSetVarProc,value=root:Packages:NIST:Event:gEvent_nslices
2779	SetVariable setvar2,pos={24,44},size={160,20},title="Max Time (s)",fSize=10
2780	SetVariable setvar2,value=root:Packages:NIST:Event:gEvent_t_longest
2781
2782	CheckBox chkbox1,pos={216,14},title="Enforce Max Time?"
2783	CheckBox chkbox1,variable = root:Packages:NIST:Event:gEvent_ForceTmaxBin
2784	Button button3,pos={500,14},size={90,20},proc=CB_SaveBinsButtonProc,title="Save Bins"
2785	Button button4,pos={500,42},size={100,20},proc=CB_ImportBinsButtonProc,title="Import Bins"
2786
2787	SetDataFolder root:Packages:NIST:Event:
2788
2789	Display/W=(291,86,706,395)/HOST=CustomBinPanel/N=BarGraph binCount vs binEndTime
2790	ModifyGraph mode=5
2791	ModifyGraph marker=19
2792	ModifyGraph lSize=2
2793	ModifyGraph rgb=(0,0,0)
2794	ModifyGraph msize=2
2795	ModifyGraph hbFill=2
2796	ModifyGraph gaps=0
2797	ModifyGraph usePlusRGB=1
2798	ModifyGraph toMode=1
2799	ModifyGraph useBarStrokeRGB=1
2800	ModifyGraph standoff=0
2801	SetAxis left 0,*
2802	Label bottom "\\Z14Time (seconds)"
2803	Label left "\\Z14Number of Events"
2804	SetActiveSubwindow ##
2805
2806	// and the table
2807	Edit/W=(13,87,280,394)/HOST=CustomBinPanel/N=T0
2808	AppendToTable/W=CustomBinPanel#T0 timeWidth,binEndTime
2809	ModifyTable width(Point)=40
2810	SetActiveSubwindow ##
2811
2812	SetDataFolder root:
2813
2814	EndMacro
2815
2816	// save the bins - use Igor Text format
2817	//
2818	Function CB_SaveBinsButtonProc(ba) : ButtonControl
2819	STRUCT WMButtonAction &ba
2820
2821	switch( ba.eventCode )
2822	case 2: // mouse up
2823	// click code here
2824
2825	SetDataFolder root:Packages:NIST:Event:
2826
2827	Wave timeWidth = timeWidth
2828	Wave binEndTime = binEndTime
2829
2830	Save/T timeWidth,binEndTime //will ask for a name
2831
2832	break
2833	case -1: // control being killed
2834	break
2835	endswitch
2836
2837	SetDataFolder root:
2838
2839	return 0
2840	End
2841
2842	// Import the bins - use Igor Text format
2843	//
2844	// -- be sure that the number of bins is reset
2845	// -?- how about the t_longest? - this should be set by the load, not here
2846	//
2847	// -- loads in timeWidth and binEndTime
2848	//
2849	Function CB_ImportBinsButtonProc(ba) : ButtonControl
2850	STRUCT WMButtonAction &ba
2851
2852	switch( ba.eventCode )
2853	case 2: // mouse up
2854	// click code here
2855	NVAR nSlice = root:Packages:NIST:Event:gEvent_nslices
2856
2857	SetDataFolder root:Packages:NIST:Event:
2858
2859	// prompt for the load of data
2860	LoadWave/T/O
2861	if(strlen(S_fileName) == 0)
2862	//user cancelled
2863	DoAlert 0,"No file selected, nothing done."
2864	return(0)
2865	endif
2866
2867	Wave timeWidth = timeWidth
2868	nSlice = numpnts(timeWidth)
2869
2870	break
2871	case -1: // control being killed
2872	break
2873	endswitch
2874
2875	SetDataFolder root:
2876
2877	return 0
2878	End
2879
2880
2881
2882	//
2883	// can either use the widths as stated -- then the end time may not
2884	// match the actual end time of the data set
2885	//
2886	// -- or --
2887	//
2888	// enforce the end time of the data set to be the end time of the bins,
2889	// then the last bin width must be reset to force the constraint
2890	//
2891	//
2892	Function CB_UpdateWavesButton(ba) : ButtonControl
2893	STRUCT WMButtonAction &ba
2894
2895	switch( ba.eventCode )
2896	case 2: // mouse up
2897	// click code here
2898	NVAR nSlice = root:Packages:NIST:Event:gEvent_nslices
2899	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
2900	NVAR enforceTmax = root:Packages:NIST:Event:gEvent_ForceTmaxBin
2901
2902	// update the waves, and recalculate everything for the display
2903	SetDataFolder root:Packages:NIST:Event:
2904
2905	Wave timeWidth = timeWidth
2906	Wave binEndTime = binEndTime
2907	Wave binCount = binCount
2908
2909	// use the widths as entered
2910	binEndTime[0] = 0
2911	binEndTime[1,] = binEndTime[p-1] + timeWidth[p-1]
2912
2913	// enforce the longest time as the end bin time
2914	// note that this changes the last time width
2915	if(enforceTmax)
2916	binEndTime[nSlice] = t_longest
2917	timeWidth[nSlice-1] = t_longest - binEndTime[nSlice-1]
2918	endif
2919
2920	binCount = p+1
2921	binCount[nSlice] = 0 // last point is zero, just for display
2922	// binCount *= sign(timeWidth) //to alert to negative time bins
2923
2924	// make the timeWidth bold and red if the widths are negative
2925	WaveStats/Q timeWidth
2926	if(V_min < 0)
2927	ModifyTable/W=CustomBinPanel#T0 style(timeWidth)=1,rgb(timeWidth)=(65535,0,0)
2928	else
2929	ModifyTable/W=CustomBinPanel#T0 style(timeWidth)=0,rgb(timeWidth)=(0,0,0)
2930	endif
2931
2932	break
2933	case -1: // control being killed
2934	break
2935	endswitch
2936
2937	SetDataFolder root:
2938
2939	return 0
2940	End
2941
2942	Function CB_HelpButtonProc(ba) : ButtonControl
2943	STRUCT WMButtonAction &ba
2944
2945	switch( ba.eventCode )
2946	case 2: // mouse up
2947	// click code here
2948	DisplayHelpTopic/Z "Event Mode Data[Setting up Custom Bin Widths]"
2949	break
2950	case -1: // control being killed
2951	break
2952	endswitch
2953
2954	return 0
2955	End
2956
2957	Function CB_Done_Proc(ba) : ButtonControl
2958	STRUCT WMButtonAction &ba
2959
2960	String win = ba.win
2961	switch (ba.eventCode)
2962	case 2:
2963	DoWindow/K CustomBinPanel
2964	break
2965	endswitch
2966	return(0)
2967	End
2968
2969
2970	Function CB_NumSlicesSetVarProc(sva) : SetVariableControl
2971	STRUCT WMSetVariableAction &sva
2972
2973	switch( sva.eventCode )
2974	case 1: // mouse up
2975	case 2: // Enter key
2976	case 3: // Live update
2977	Variable dval = sva.dval
2978	String sval = sva.sval
2979	SetDataFolder root:Packages:NIST:Event:
2980
2981	Wave timeWidth = timeWidth
2982	Wave binEndTime = binEndTime
2983
2984	Redimension/N=(dval) timeWidth
2985	Redimension/N=(dval+1) binEndTime,binCount
2986
2987	SetDataFolder root:
2988
2989	break
2990	case -1: // control being killed
2991	break
2992	endswitch
2993
2994	return 0
2995	End
2996
2997
2998	///////////////////
2999	//
3000	// utility to split a large file
3001	// 100 MB is the recommended size
3002	// events can be clipped here, so be sure to trim the ends of the
3003	// resulting files as needed.
3004	//
3005	// - works like the unix 'split' command
3006	//
3007	//
3008
3009	Proc SplitBigFile(splitSize, baseStr)
3010	Variable splitSize = 100
3011	String baseStr="split"
3012	Prompt splitSize,"Target file size, in MB (< 150 MB!!)"
3013	Prompt baseStr,"File prefix, number will be appended"
3014
3015	if(splitSize > root:Packages:NIST:Event:gEventFileTooLarge)
3016	Abort "File split must be less than 150 MB. Please try again"
3017	endif
3018
3019	fSplitBigFile(splitSize, baseStr)
3020
3021	ShowSplitFileTable()
3022	End
3023
3024	Function/S fSplitBigFile(splitSize, baseStr)
3025	Variable splitSize
3026	String baseStr
3027
3028
3029	String fileName="" // File name, partial path, full path or "" for dialog.
3030	Variable refNum
3031	String str
3032	SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3033
3034	listStr="" //initialize output list
3035
3036	Variable readSize=1e6 //1 MB
3037	Make/O/B/U/N=(readSize) aBlob //1MB worth
3038	Variable numSplit
3039	Variable num,ii,jj,outRef,frac
3040	String thePath, outStr
3041
3042	Printf "SplitSize = %u MB\r",splitSize
3043	splitSize = trunc(splitSize) * 1e6 // now in bytes
3044
3045
3046	// Open file for read.
3047	Open/R/Z=2/F="????"/P=catPathName refNum as fileName
3048	thePath = ParseFilePath(1, S_fileName, ":", 1, 0)
3049	Print "thePath = ",thePath
3050
3051	// Store results from Open in a safe place.
3052	Variable err = V_flag
3053	String fullPath = S_fileName
3054
3055	if (err == -1)
3056	Print "cancelled by user."
3057	return ("")
3058	endif
3059
3060	FStatus refNum
3061
3062	Printf "total # bytes = %u\r",V_logEOF
3063
3064	numSplit=0
3065	if(V_logEOF > splitSize)
3066	numSplit = trunc(V_logEOF/splitSize)
3067	endif
3068
3069	frac = V_logEOF - numSplit*splitSize
3070	Print "numSplit = ",numSplit
3071	Printf "frac = %u\r",frac
3072
3073	num=0
3074	if(frac > readSize)
3075	num = trunc(frac/readSize)
3076	endif
3077
3078
3079	frac = frac - num*readSize
3080
3081	Print "num = ",num
3082	Printf "frac = %u\r",frac
3083
3084	// baseStr = "split"
3085
3086	for(ii=0;ii<numSplit;ii+=1)
3087	outStr = (thePath+baseStr+num2str(ii)+".hst")
3088	// Print "outStr = ",outStr
3089	Open outRef as outStr
3090
3091	for(jj=0;jj<(splitSize/readSize);jj+=1)
3092	FBinRead refNum,aBlob
3093	FBinWrite outRef,aBlob
3094	endfor
3095
3096	Close outRef
3097	// listStr += outStr+";" //no, this is the FULL path
3098	listStr += baseStr+num2str(ii)+".hst"+";"
3099	endfor
3100
3101	Make/O/B/U/N=(frac) leftover
3102	// ii was already incremented past the loop
3103	outStr = (thePath+baseStr+num2str(ii)+".hst")
3104	Open outRef as outStr
3105	for(jj=0;jj<num;jj+=1)
3106	FBinRead refNum,aBlob
3107	FBinWrite outRef,aBlob
3108	endfor
3109	FBinRead refNum,leftover
3110	FBinWrite outRef,leftover
3111
3112	Close outRef
3113	// listStr += outStr+";"
3114	listStr += baseStr+num2str(ii)+".hst"+";"
3115
3116	FSetPos refNum,V_logEOF
3117	Close refNum
3118
3119	KillWaves/Z aBlob,leftover
3120	return(listStr)
3121	End
3122
3123	// allows the list of loaded files to be edited
3124	Function ShowSplitFileTable()
3125
3126	SVAR str = root:Packages:NIST:Event:gSplitFileList
3127
3128	WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3129	if(waveExists(tw) != 1)
3130	Make/O/T/N=1 root:Packages:NIST:Event:SplitFileWave
3131	WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3132	endif
3133
3134	List2TextWave(str,tw)
3135	Edit tw
3136
3137	return(0)
3138	End
3139
3140
3141	//// save the sliced data, and accumulate slices
3142	// *** this works with sliced data -- that is data that has been PROCESSED
3143	//
3144	// need some way of ensuring that the slices match up since I'm blindly adding them together.
3145	//
3146	// mode = 0 wipe out the old accumulated, copy slicedData to accumulatedData
3147	// mode = 1 add current slicedData to accumulatedData
3148	// mode = 2 copy accumulatedData to slicedData in preparation of export or display
3149	// mode = 3 unused...
3150	//
3151	// "Split Large File",SplitBigFile()
3152	// "Accumulate First Slice",AccumulateSlices(0)
3153	// "Add Current Slice",AccumulateSlices(1)
3154	// "Display Accumulated Slices",AccumulateSlices(2)
3155	//
3156	Function AccumulateSlicesButton(ctrlName) : ButtonControl
3157	String ctrlName
3158
3159	Variable mode
3160	mode = str2num(ctrlName[strlen(ctrlName)-1])
3161	// Print "mode=",mode
3162	AccumulateSlices(mode)
3163
3164	return(0)
3165	End
3166
3167	// added an extra (optional) parameter for silent operation - to be used when a list of
3168	// files are processed - so the user can walk away as it works. Using the buttons from the
3169	// panel will still present the alerts
3170	//
3171	Function AccumulateSlices(mode,[skipAlert])
3172	Variable mode,skipAlert
3173
3174	if(ParamIsDefault(skipAlert))
3175	skipAlert = 0
3176	endif
3177	SetDataFolder root:Packages:NIST:Event:
3178
3179	switch(mode)
3180	case 0:
3181	if(!skipAlert)
3182	DoAlert 0,"The current data has been copied to the accumulated set. You are now ready to add more data."
3183	endif
3184	KillWaves/Z accumulatedData
3185	Duplicate/O slicedData accumulatedData
3186	KillWaves/Z accum_BinCount // the accumulated BinCount
3187	Duplicate/O binCount accum_binCount
3188	break
3189	case 1:
3190	if(!skipAlert)
3191	DoAlert 0,"The current data has been added to the accumulated data. You can add more data."
3192	endif
3193	Wave acc=accumulatedData
3194	Wave cur=slicedData
3195	acc += cur
3196	Wave binCount = binCount
3197	Wave accum_binCount = accum_BinCount
3198	accum_BinCount += binCount
3199	binCount = accum_binCount // !! NOTE - the bin count wave is now the total
3200	// AccumulateSlices(2,skipAlert=skipAlert) // and a recursive call to show the total!
3201	break
3202	case 2:
3203	if(!skipAlert)
3204	DoAlert 0,"The accumulated data is now the display data and is ready for display or export."
3205	endif
3206	Duplicate/O accumulatedData slicedData
3207	// do something to "touch" the display to force it to update
3208	NVAR gLog = root:Packages:NIST:Event:gEvent_logint
3209	LogIntEvent_Proc("",gLog)
3210	break
3211	default:
3212
3213	endswitch
3214
3215	SetDataFolder root:
3216	return(0)
3217	end
3218
3219
3220	////////////////////////////////////////////
3221	//
3222	// Panel and procedures for decimation
3223	//
3224	////////////////////////////////////////////
3225
3226	//Function E_ShowDecimateButton(ctrlName) : ButtonControl
3227	// String ctrlName
3228	//
3229	// DoWindow/F DecimatePanel
3230	// if(V_flag ==0)
3231	// Execute "DecimatePanel()"
3232	// endif
3233	//End
3234	//
3235	//
3236	//Proc DecimatePanel() //: Panel
3237	//
3238	// PauseUpdate; Silent 1 // building window...
3239	// NewPanel /W=(1602,44,1961,380)/K=1
3240	//// ShowTools/A
3241	// Button button0,pos={29,15},size={100,20},proc=SplitFileButtonProc,title="Split Big File"
3242	// SetVariable setvar0,pos={182,55},size={150,15},title="Decimation factor",fsize=10
3243	// SetVariable setvar0,limits={1,inf,1},value= root:Packages:NIST:Event:gDecimation
3244	// Button button1,pos={26,245},size={150,20},proc=LoadDecimateButtonProc,title="Load and Decimate"
3245	// Button button2,pos={25,277},size={150,20},proc=ConcatenateButtonProc,title="Concatenate"
3246	// Button button3,pos={25,305},size={150,20},proc=DisplayConcatenatedButtonProc,title="Display Concatenated"
3247	// Button button4,pos={29,52},size={130,20},proc=Stream_LoadDecim,title="Load From List"
3248	//
3249	// GroupBox group0 title="Manual Controls",size={185,112},pos={14,220}
3250	//EndMacro
3251
3252
3253	Function SplitFileButtonProc(ctrlName) : ButtonControl
3254	String ctrlName
3255
3256	Execute "SplitBigFile()"
3257	End
3258
3259
3260	// show all of the data
3261	//
3262	Proc ShowDecimatedGraph()
3263
3264	DoWindow/F DecimatedGraph
3265	if(V_flag == 0)
3266	PauseUpdate; Silent 1 // building window...
3267	String fldrSav0= GetDataFolder(1)
3268	SetDataFolder root:Packages:NIST:Event:
3269	Display /W=(25,44,486,356)/K=1/N=DecimatedGraph rescaledTime_dec
3270	SetDataFolder fldrSav0
3271	ModifyGraph mode=4
3272	ModifyGraph marker=19
3273	ModifyGraph rgb(rescaledTime_dec)=(0,0,0)
3274	ModifyGraph msize=1
3275	ErrorBars rescaledTime_dec OFF
3276	Label left "\\Z14Time (seconds)"
3277	Label bottom "\\Z14Event number"
3278	ShowInfo
3279	endif
3280
3281	EndMacro
3282
3283	// data has NOT been processed
3284	//
3285	// so work with x,y,t, and rescaled time
3286	// variables -- t_longest
3287	Function ConcatenateButtonProc(ctrlName) : ButtonControl
3288	String ctrlName
3289
3290	DoAlert 1,"Is this the first file?"
3291	Variable first = V_flag
3292
3293	fConcatenateButton(first)
3294
3295	return(0)
3296	End
3297
3298	Function fConcatenateButton(first)
3299	Variable first
3300
3301
3302	SetDataFolder root:Packages:NIST:Event:
3303
3304	Wave timePt_dTmp=timePt_dTmp
3305	Wave xLoc_dTmp=xLoc_dTmp
3306	Wave yLoc_dTmp=yLoc_dTmp
3307	Wave rescaledTime_dTmp=rescaledTime_dTmp
3308
3309	NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3310	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3311
3312
3313	if(first==1) //1==yes, 2==no
3314	//then copy the files over, adjusting the time to start from zero
3315	// rescaledTime starts from zero (set by the loader)
3316
3317	timePt_dTmp -= timePt_dTmp[0] //subtract the first value
3318
3319	Duplicate/O timePt_dTmp timePt_dec
3320	Duplicate/O xLoc_dTmp xLoc_dec
3321	Duplicate/O yLoc_dTmp yLoc_dec
3322	Duplicate/O rescaledTime_dTmp rescaledTime_dec
3323
3324	t_longest_dec = t_longest
3325
3326	else
3327	// concatenate the files + adjust the time
3328	Wave timePt_dec=timePt_dec
3329	Wave xLoc_dec=xLoc_dec
3330	Wave yLoc_dec=yLoc_dec
3331	Wave rescaledTime_dec=rescaledTime_dec
3332
3333	// adjust the times -- assuming they add
3334	// rescaledTime starts from zero (set by the loader)
3335	//
3336	//
3337	rescaledTime_dTmp += rescaledTime_dec[numpnts(rescaledTime_dec)-1]
3338	rescaledTime_dTmp += abs(rescaledTime_dec[numpnts(rescaledTime_dec)-1] - rescaledTime_dec[numpnts(rescaledTime_dec)-2])
3339
3340	timePt_dTmp -= timePt_dTmp[0] //subtract the first value
3341
3342	timePt_dTmp += timePt_dec[numpnts(timePt_dec)-1] // offset by the last point
3343	timePt_dTmp += abs(timePt_dec[numpnts(timePt_dec)-1] - timePt_dec[numpnts(timePt_dec)-2]) // plus delta so there's not a flat step
3344
3345	Concatenate/NP/O {timePt_dec,timePt_dTmp}, tmp
3346	Duplicate/O tmp timePt_dec
3347
3348	Concatenate/NP/O {xLoc_dec,xLoc_dTmp}, tmp
3349	Duplicate/O tmp xLoc_dec
3350
3351	Concatenate/NP/O {yLoc_dec,yLoc_dTmp}, tmp
3352	Duplicate/O tmp yLoc_dec
3353
3354	Concatenate/NP/O {rescaledTime_dec,rescaledTime_dTmp}, tmp
3355	Duplicate/O tmp rescaledTime_dec
3356
3357
3358	KillWaves tmp
3359
3360	t_longest_dec = rescaledTime_dec[numpnts(rescaledTime_dec)-1]
3361
3362	endif
3363
3364
3365	SetDataFolder root:
3366
3367	return(0)
3368
3369	End
3370
3371	Function DisplayConcatenatedButtonProc(ctrlName) : ButtonControl
3372	String ctrlName
3373
3374	//copy the files over to the display set for processing
3375	SetDataFolder root:Packages:NIST:Event:
3376
3377	Wave timePt_dec=timePt_dec
3378	Wave xLoc_dec=xLoc_dec
3379	Wave yLoc_dec=yLoc_dec
3380	Wave rescaledTime_dec=rescaledTime_dec
3381
3382	NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3383	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3384
3385	Duplicate/O timePt_dec timePt
3386	Duplicate/O xLoc_dec xLoc
3387	Duplicate/O yLoc_dec yLoc
3388	Duplicate/O rescaledTime_dec rescaledTime
3389
3390	t_longest = t_longest_dec
3391
3392	SetDataFolder root:
3393
3394	return(0)
3395
3396	End
3397
3398
3399	Macro DisplayForSlicing()
3400
3401	// plot the EventBarGraph?
3402	SetDataFolder root:Packages:NIST:Event:
3403	Display /W=(110,705,610,1132)/N=SliceGraph /K=1 binCount vs binEndTime
3404	ModifyGraph mode=5
3405	ModifyGraph marker=19
3406	ModifyGraph lSize=2
3407	ModifyGraph rgb=(0,0,0)
3408	ModifyGraph msize=2
3409	ModifyGraph hbFill=2
3410	ModifyGraph gaps=0
3411	ModifyGraph usePlusRGB=1
3412	ModifyGraph toMode=1
3413	ModifyGraph useBarStrokeRGB=1
3414	ModifyGraph standoff=0
3415
3416	SetDataFolder root:
3417
3418	// append the differential
3419	AppendToGraph/R root:Packages:NIST:Event:rescaledTime_DIF vs root:Packages:NIST:Event:rescaledTime
3420	ModifyGraph rgb(rescaledTime_DIF)=(1,16019,65535)
3421
3422	End
3423
3424
3425
3426	// unused, old testing procedure
3427	Function LoadDecimateButtonProc(ctrlName) : ButtonControl
3428	String ctrlName
3429
3430	LoadEventLog_Button("")
3431
3432	// now decimate
3433	SetDataFolder root:Packages:NIST:Event:
3434
3435	Wave timePt=timePt
3436	Wave xLoc=xLoc
3437	Wave yLoc=yLoc
3438	NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3439
3440	NVAR decimation = root:Packages:NIST:Event:gDecimation
3441
3442
3443	Duplicate/O timePt, timePt_dTmp
3444	Duplicate/O xLoc, xLoc_dTmp
3445	Duplicate/O yLoc, yLoc_dTmp
3446	Resample/DOWN=(decimation)/N=1 timePt_dTmp
3447	Resample/DOWN=(decimation)/N=1 xLoc_dTmp
3448	Resample/DOWN=(decimation)/N=1 yLoc_dTmp
3449
3450
3451	Duplicate/O timePt_dTmp rescaledTime_dTmp
3452	rescaledTime_dTmp = 1e-7*(timePt_dTmp - timePt_dTmp[0]) //convert to seconds and start from zero
3453	t_longest_dec = waveMax(rescaledTime_dTmp) //should be the last point
3454
3455	SetDataFolder root:
3456
3457
3458	End
3459
3460
3461
3462
3463
3464
3465
3466	//
3467	// loads a list of files, decimating each chunk as it is read in
3468	//
3469	Function Stream_LoadDecim(ctrlName)
3470	String ctrlName
3471
3472	Variable fileref
3473	Variable num,ii
3474
3475	SVAR filename = root:Packages:NIST:Event:gEvent_logfile
3476	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3477
3478	SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3479	NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3480	NVAR decimation = root:Packages:NIST:Event:gDecimation
3481
3482	String pathStr
3483	PathInfo catPathName
3484	pathStr = S_Path
3485
3486	// if "stream" mode is not checked - abort
3487	NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
3488	if(gEventModeRadioVal != MODE_STREAM)
3489	Abort "The mode must be 'Stream' to use this function"
3490	return(0)
3491	endif
3492
3493	// if the list has been edited, turn it into a list
3494	WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3495	if(WaveExists(tw))
3496	listStr = TextWave2SemiList(tw)
3497	else
3498	ShowSplitFileTable()
3499	DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3500	return(0)
3501	endif
3502
3503
3504	//loop through everything in the list
3505	num = ItemsInList(listStr)
3506	if(num == 0)
3507	DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3508	return (0)
3509	endif
3510
3511	for(ii=0;ii<num;ii+=1)
3512
3513	// (1) load the file, prepending the path
3514	filename = pathStr + StringFromList(ii, listStr ,";")
3515	if(EventDataType(filename) != 1) //if not hst extension of raw data
3516	Abort "Data must be raw event data with file extension .hst for this operation"
3517	return(0)
3518	endif
3519
3520	#if (exists("EventLoadWave")==4)
3521	LoadEvents_XOP()
3522	#else
3523	LoadEvents()
3524	#endif
3525
3526	SetDataFolder root:Packages:NIST:Event: //LoadEvents sets back to root:
3527
3528	Wave timePt=timePt
3529	Wave xLoc=xLoc
3530	Wave yLoc=yLoc
3531	CleanupTimes(xLoc,yLoc,timePt) //remove zeroes
3532
3533	Duplicate/O timePt rescaledTime
3534	rescaledTime = 1e-7*(timePt-timePt[0]) //convert to seconds and start from zero
3535	t_longest = waveMax(rescaledTime) //should be the last point
3536
3537	// (2) do the decimation, just on timePt. create rescaledTime from the decimated timePt
3538
3539	Duplicate/O timePt, timePt_dTmp
3540	Duplicate/O xLoc, xLoc_dTmp
3541	Duplicate/O yLoc, yLoc_dTmp
3542	Resample/DOWN=(decimation)/N=1 timePt_dTmp
3543	Resample/DOWN=(decimation)/N=1 xLoc_dTmp
3544	Resample/DOWN=(decimation)/N=1 yLoc_dTmp
3545
3546
3547	Duplicate/O timePt_dTmp rescaledTime_dTmp
3548	rescaledTime_dTmp = 1e-7*(timePt_dTmp - timePt_dTmp[0]) //convert to seconds and start from zero
3549	t_longest_dec = waveMax(rescaledTime_dTmp) //should be the last point
3550
3551
3552	// (3) concatenate the files + adjust the time
3553	fConcatenateButton(ii+1) //passes 1 for the first time, >1 each other time
3554
3555	endfor
3556
3557	//// Now that everything is decimated and concatenated, create the rescaled time wave
3558	// SetDataFolder root:Packages:NIST:Event: //LoadEvents sets back to root:
3559	// Wave timePt_dec = timePt_dec
3560	// Duplicate/O timePt_dec rescaledTime_dec
3561	// rescaledTime_dec = 1e-7*(timePt_dec - timePt_dec[0]) //convert to seconds and start from zero
3562	// t_longest_dec = waveMax(rescaledTime_dec) //should be the last point
3563
3564	DisplayConcatenatedButtonProc("")
3565
3566	SetDataFolder root:
3567
3568	return(0)
3569	End
3570
3571	Function ShowList_ToLoad(ctrlName)
3572	String ctrlName
3573
3574	ShowSplitFileTable()
3575
3576	return(0)
3577	End
3578
3579
3580	//
3581	// loads a list of files that have been adjusted and saved
3582	// -- does not decimate
3583	//
3584	Function Stream_LoadAdjustedList(ctrlName)
3585	String ctrlName
3586
3587	Variable fileref
3588
3589	SVAR filename = root:Packages:NIST:Event:gEvent_logfile
3590	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3591
3592	SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3593	NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3594	// NVAR decimation = root:Packages:NIST:Event:gDecimation
3595
3596	String pathStr
3597	PathInfo catPathName
3598	pathStr = S_Path
3599
3600	// if "stream" mode is not checked - abort
3601	NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
3602	if(gEventModeRadioVal != MODE_STREAM)
3603	Abort "The mode must be 'Stream' to use this function"
3604	return(0)
3605	endif
3606
3607	// if the list has been edited, turn it into a list
3608	WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3609	if(WaveExists(tw))
3610	listStr = TextWave2SemiList(tw)
3611	else
3612	ShowSplitFileTable()
3613	DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3614	return(0)
3615	endif
3616
3617
3618	//loop through everything in the list
3619	Variable num,ii
3620	num = ItemsInList(listStr)
3621
3622	if(num == 0)
3623	DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3624	return(0)
3625	endif
3626
3627	for(ii=0;ii<num;ii+=1)
3628
3629	// (1) load the file, prepending the path
3630	filename = pathStr + StringFromList(ii, listStr ,";")
3631	if(EventDataType(filename) != 2) //if not itx extension of edited data
3632	Abort "Data must be edited data with file extension .itx for this operation"
3633	return(0)
3634	endif
3635	SetDataFolder root:Packages:NIST:Event:
3636	LoadWave/T/O fileName
3637
3638	SetDataFolder root:Packages:NIST:Event: //LoadEvents sets back to root: ??
3639
3640	// this is what is loaded -- _dec extension is what is concatenated, and will be copied back later
3641	Wave timePt=timePt
3642	Wave xLoc=xLoc
3643	Wave yLoc=yLoc
3644	Wave rescaledTime=rescaledTime
3645
3646	// CleanupTimes(xLoc,yLoc,timePt) //remove zeroes
3647
3648	// Duplicate/O timePt rescaledTime
3649	// rescaledTime = 1e-7*(timePt-timePt[0]) //convert to seconds and start from zero
3650	// t_longest = waveMax(rescaledTime) //should be the last point
3651
3652	// (2) No decimation
3653
3654	Duplicate/O timePt, timePt_dTmp
3655	Duplicate/O xLoc, xLoc_dTmp
3656	Duplicate/O yLoc, yLoc_dTmp
3657	Duplicate/O rescaledTime, rescaledTime_dTmp
3658
3659
3660	// (3) concatenate the files + adjust the time
3661	fConcatenateButton(ii+1) //passes 1 for the first time, >1 each other time
3662
3663	endfor
3664
3665	DisplayConcatenatedButtonProc("") // this resets the longest time, too
3666
3667	SetDataFolder root:
3668
3669	return(0)
3670	End
3671
3672	//
3673	// loads a list of files that have been adjusted and saved
3674	// -- does not decimate
3675	// -- bins as they are loaded
3676	//
3677	Function Stream_LoadAdjList_BinOnFly(ctrlName)
3678	String ctrlName
3679
3680	Variable fileref
3681
3682	SVAR filename = root:Packages:NIST:Event:gEvent_logfile
3683	SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3684
3685	String pathStr
3686	PathInfo catPathName
3687	pathStr = S_Path
3688
3689	// if "stream" mode is not checked - abort
3690	NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
3691	if(gEventModeRadioVal != MODE_STREAM)
3692	Abort "The mode must be 'Stream' to use this function"
3693	return(0)
3694	endif
3695
3696	// if the list has been edited, turn it into a list
3697	WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3698	if(WaveExists(tw))
3699	listStr = TextWave2SemiList(tw)
3700	else
3701	ShowSplitFileTable()
3702	DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3703	return(0)
3704	endif
3705
3706
3707	//loop through everything in the list
3708	Variable num,ii,jj
3709	num = ItemsInList(listStr)
3710	if(num == 0)
3711	DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3712	return(0)
3713	endif
3714
3715	////////////////
3716	// declarations for the binning
3717	// for the first pass, do all of this
3718	Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
3719
3720	Wave binnedData = root:Packages:NIST:Event:binnedData
3721	Wave xLoc = root:Packages:NIST:Event:xLoc
3722	Wave yLoc = root:Packages:NIST:Event:yLoc
3723
3724	// now with the number of slices and max time, process the events
3725
3726	NVAR yesSortStream = root:Packages:NIST:Event:gSortStreamEvents //do I sort the events?
3727	NVAR t_longest = root:Packages:NIST:Event:gEvent_t_longest
3728	NVAR nslices = root:Packages:NIST:Event:gEvent_nslices
3729	// NVAR t_longest_dec = root:Packages:NIST:Event:gEvent_t_longest_decimated
3730	NVAR t_segment_start = root:Packages:NIST:Event:gEvent_t_segment_start
3731	NVAR t_segment_end = root:Packages:NIST:Event:gEvent_t_segment_end
3732
3733	SetDataFolder root:Packages:NIST:Event //don't count on the folder remaining here
3734
3735	Make/D/O/N=(128,128,nslices) slicedData
3736
3737	Wave slicedData = slicedData
3738	Wave rescaledTime = rescaledTime
3739	Make/O/D/N=(128,128) tmpData
3740	Make/O/D/N=(nslices+1) binEndTime,binCount//,binStartTime
3741	Make/O/D/N=(nslices) timeWidth
3742	Wave binEndTime = binEndTime
3743	Wave timeWidth = timeWidth
3744	Wave binCount = binCount
3745
3746	slicedData = 0
3747	binCount = 0
3748
3749	variable p1,p2,t1,t2
3750
3751	t_segment_start = 0
3752	// start w/file 1??
3753	for(jj=0;jj<num;jj+=1)
3754
3755	// (1) load the file, prepending the path
3756	filename = pathStr + StringFromList(jj, listStr ,";")
3757	if(EventDataType(filename) != 2) //if not itx extension of edited data
3758	Abort "Data must be edited data with file extension .itx for this operation"
3759	return(0)
3760	endif
3761	SetDataFolder root:Packages:NIST:Event:
3762	LoadWave/T/O fileName
3763
3764	SetDataFolder root:Packages:NIST:Event: //LoadEvents sets back to root: ??
3765
3766	// this is what is loaded --
3767	Wave timePt=timePt
3768	Wave xLoc=xLoc
3769	Wave yLoc=yLoc
3770	Wave rescaledTime=rescaledTime
3771
3772	if(jj==0)
3773	timePt -= timePt[0] //make sure the first point is zero
3774	t_segment_start = 0
3775	t_segment_end = rescaledTime[numpnts(rescaledTime)-1]
3776	else
3777	t_segment_start = t_segment_end
3778	t_segment_end += rescaledTime[numpnts(rescaledTime)-1]
3779
3780	rescaledTime += t_segment_start
3781	endif
3782
3783	Printf "jj=%g\t\tt_segment_start=%g\tt_segment_end=%g\r",jj,t_segment_start,t_segment_end
3784
3785	// this is the binning--
3786
3787	// del = t_longest/nslices
3788
3789	// TODO
3790	// the global exists for this switch, but it is not implemented - not sure whether
3791	// it's correct to implement this at all --
3792	//
3793	if(yesSortStream == 1)
3794	SortTimeData()
3795	endif
3796
3797	// index the events before binning
3798	// if there is a sort of these events, I need to re-index the events for the histogram
3799	// SetDataFolder root:Packages:NIST:Event
3800	IndexForHistogram(xLoc,yLoc,binnedData)
3801	// SetDataFolder root:
3802	Wave index = root:Packages:NIST:Event:SavedIndex //the index for the histogram
3803
3804	for(ii=0;ii<nslices;ii+=1)
3805	if(ii==0)
3806	// t1 = ii*del
3807	// t2 = (ii+1)*del
3808	p1 = BinarySearch(rescaledTime,0)
3809	p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
3810	else
3811	// t2 = (ii+1)*del
3812	p1 = p2+1 //one more than the old one
3813	p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
3814	endif
3815
3816	if(p1 == -1)
3817	Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
3818	p1 = 0 //set to the first point if it's off the end
3819	Endif
3820	if(p2 == -2)
3821	Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
3822	p2 = numpnts(rescaledTime)-1 //set to the last point if it's off the end
3823	Endif
3824	// Print p1,p2
3825
3826	tmpData=0
3827	JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
3828	// slicedData[][][ii] = tmpData[p][q]
3829	slicedData[][][ii] += tmpData[p][q]
3830
3831	// Duplicate/O tmpData,$("tmpData_"+num2str(ii)+"_"+num2str(jj))
3832
3833	// binEndTime[ii+1] = t2
3834	// binCount[ii] = sum(tmpData,-inf,inf)
3835	binCount[ii] += sum(tmpData,-inf,inf)
3836	endfor
3837
3838	endfor //end loop over the file list
3839
3840	// after all of the files have been loaded
3841	t_longest = t_segment_end //this is the total time
3842
3843	Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
3844	Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
3845	logslicedData = log(slicedData)
3846
3847
3848	SetDataFolder root:
3849
3850	return(0)
3851	End
3852
3853
3854	/////////////////////////////////////
3855
3856	// dd-mon-yyyy hh:mm:ss -> Event file name
3857	// the VAX uses 24 hr time for hh
3858	//
3859	// scans as string elements since I'm reconstructing a string name
3860	Function/S DateAndTime2HSTName(dateandtime)
3861	string dateAndTime
3862
3863	String day,yr,hh,mm,ss,time_secs
3864	Variable mon
3865	string str,monStr,fileStr
3866
3867	str=dateandtime
3868	sscanf str,"%2s-%3s-%4s %2s:%2s:%2s",day,monStr,yr,hh,mm,ss
3869	mon = monStr2num(monStr)
3870
3871	fileStr = "Event"+yr+num2str(mon)+day+hh+mm+ss+".hst"
3872	Print fileStr
3873
3874	return(fileStr)
3875	end
3876
3877	// dd-mon-yyyy hh:mm:ss -> Event file name
3878	// the VAX uses 24 hr time for hh
3879	//
3880	// scans as string elements since I'm reconstructing a string name
3881	Function DateAndTime2HSTNumber(dateandtime)
3882	string dateAndTime
3883
3884	String day,yr,hh,mm,ss,time_secs
3885	Variable mon,num
3886	string str,monStr,fileStr
3887
3888	str=dateandtime
3889	sscanf str,"%2s-%3s-%4s %2s:%2s:%2s",day,monStr,yr,hh,mm,ss
3890	mon = monStr2num(monStr)
3891
3892	fileStr = yr+num2str(mon)+day+hh+mm+ss
3893	num = str2num(fileStr)
3894
3895	return(num)
3896	end
3897
3898	Function HSTName2Num(str)
3899	String str
3900
3901	Variable num
3902	sscanf str,"Event%d.hst",num
3903	return(num)
3904	end
3905	/////////////////////////////
3906
3907	Proc GetListofITXorSplitFiles(searchStr)
3908	String searchStr = ".itx"
3909	ShowList_ToLoad("")
3910	GetListofITXHST(searchStr)
3911	end
3912
3913	// needs SANS_Utilites.ipf
3914	Function GetListofITXHST(searchStr)
3915	String searchStr
3916
3917	String fullList = IndexedFile(catpathName, -1, "????" )
3918
3919	// list is currently everything -- trim it
3920	searchStr = "" + searchStr + ""
3921	String list = ListMatch(fullList, searchStr )
3922
3923	Print "searchString = ",searchStr
3924	//
3925	Wave/T tw = root:Packages:NIST:Event:SplitFileWave
3926	List2TextWave(list,tw)
3927	Sort/A tw,tw //alphanumeric sort
3928	Edit tw
3929	return(0)
3930	End
3931
3932	//
3933	// loads a list of files that have been adjusted and saved
3934	// -- MUST run one through first manually to set the bin sapcing, then this
3935	// won't touch that, just process all of the files
3936	// -- bins as they are loaded
3937	//
3938	Function Osc_LoadAdjList_BinOnFly(ctrlName)
3939	String ctrlName
3940
3941	Variable fileref
3942
3943	SVAR filename = root:Packages:NIST:Event:gEvent_logfile
3944	SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3945
3946	String pathStr,fileStr,tmpStr
3947	PathInfo catPathName
3948	pathStr = S_Path
3949
3950	// if "Osc" mode is not checked - abort
3951	NVAR gEventModeRadioVal= root:Packages:NIST:Event:gEvent_mode
3952	if(gEventModeRadioVal != MODE_OSCILL)
3953	Abort "The mode must be 'Oscillatory' to use this function"
3954	return(0)
3955	endif
3956
3957	// if the list has been edited, turn it into a list
3958	WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3959	if(WaveExists(tw))
3960	listStr = TextWave2SemiList(tw)
3961	else
3962	ShowSplitFileTable()
3963	DoAlert 0,"Enter the file names in the table, then click 'Load From List' again."
3964	return(0)
3965	endif
3966
3967
3968	//loop through everything in the list
3969	Variable num,ii,jj
3970	num = ItemsInList(listStr)
3971
3972	//handle the first file differently
3973	//(1) load (import edited)
3974	jj=0
3975	filename = pathStr + StringFromList(jj, listStr ,";")
3976	if(EventDataType(filename) != 2) //if not itx extension of edited data
3977	Abort "Data must be edited data with file extension .itx for this operation"
3978	return(0)
3979	endif
3980
3981	SetDataFolder root:Packages:NIST:Event:
3982	LoadWave/T/O fileName
3983	SetDataFolder root:Packages:NIST:Event: //LoadEvents sets back to root: ??
3984
3985	// clear out the old sort index, if present, since new data is being loaded
3986	KillWaves/Z OscSortIndex
3987	SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
3988	sprintf tmpStr, "%s: a user-modified event file\r",StringFromList(jj, listStr ,";")
3989	dispStr = tmpStr
3990
3991	//(2) bin
3992	Osc_ProcessEventLog("")
3993
3994	//(3) display bin details
3995	Execute "ShowBinTable()"
3996	Execute "BinEventBarGraph()"
3997
3998	//(4) add first
3999	AccumulateSlices(0,skipAlert=1) //bypass the dialog
4000
4001	// with remaining files
4002	// load
4003	// bin
4004	// add next
4005
4006	// start w/file 1, since "0th" file was already processed
4007	for(jj=1;jj<num;jj+=1)
4008
4009	//(1) load (import edited)
4010	filename = pathStr + StringFromList(jj, listStr ,";")
4011	if(EventDataType(filename) != 2) //if not itx extension of edited data
4012	Abort "Data must be edited data with file extension .itx for this operation"
4013	return(0)
4014	endif
4015	SetDataFolder root:Packages:NIST:Event:
4016	LoadWave/T/O fileName
4017	SetDataFolder root:Packages:NIST:Event: //LoadEvents sets back to root: ??
4018
4019	// clear out the old sort index, if present, since new data is being loaded
4020	KillWaves/Z OscSortIndex
4021	SVAR dispStr = root:Packages:NIST:Event:gEventDisplayString
4022	sprintf tmpStr, "%s: a user-modified event file\r",StringFromList(jj, listStr ,";")
4023	dispStr = tmpStr
4024
4025	//(2) bin
4026	Osc_ProcessEventLog("")
4027
4028	//(3) display bin details
4029	Execute "ShowBinTable()"
4030	Execute "BinEventBarGraph()"
4031
4032	//(4) add next
4033	AccumulateSlices(1,skipAlert=1)
4034
4035	endfor //end loop over the file list
4036
4037
4038	//display total
4039	AccumulateSlices(2,skipAlert=1)
4040
4041	// let user know we're done
4042	DoAlert 0,"The list has been processed and is ready for export"
4043	SetDataFolder root:
4044
4045	return(0)
4046	End
4047
4048
4049	//returns 0 for unknown data
4050	// 1 for raw hst files
4051	// 2 for edited ITX files
4052	//
4053	// TODO
4054	// -- currently, this blindly assumes an .hst extension for raw HST data, and .itx for edited data
4055	//
4056	Function EventDataType(filename)
4057	string filename
4058
4059	Variable type
4060	Variable offset = strsearch(filename,".",Inf,1) //work backwards
4061	String extension = filename[offset+1,strlen(filename)-1]
4062	//Print extension
4063	type = 0
4064	if(cmpstr(extension,"hst") == 0)
4065	type = 1
4066	endif
4067	if(cmpstr(extension,"itx") == 0)
4068	type = 2
4069	endif
4070	return(type)
4071	end
4072
4073	////////////
4074	// this could be jazzed up quite a bit, but it is a first pass at
4075	// figuring out what is going on with the events
4076	//
4077	Proc DecodeBinaryEvents(numToPrint)
4078	Variable numToPrint=10
4079
4080	DecodeEvents(numToPrint)
4081	end
4082
4083	// sscanf %x takes hex input and gives a real value output
4084	// printf %d val gives the integer representation of the sscanf output
4085	// printf %b val gives the binary representation.
4086	Function DecodeEvents(numToPrint)
4087	Variable numToPrint
4088
4089	String filePathStr,buffer
4090	Variable fileRef,type,dataval,num0,num1,num2,num3
4091	Variable numT0,numPP,numZero,numXYevents,bit29
4092	Variable ii
4093
4094	ii=0
4095	Open/R fileref
4096	tic()
4097	do
4098	do
4099	FReadLine fileref, buffer //skip the "blank" lines that have one character
4100	while(strlen(buffer) == 1)
4101
4102	if (strlen(buffer) == 0)
4103	break
4104	endif
4105
4106	sscanf buffer,"%x",dataval
4107	if(ii<numToPrint)
4108	printf "%s %0.32b ",buffer[0,7],dataval
4109	endif
4110
4111	// two most sig bits (31-30)
4112	type = (dataval & 0xC0000000)/1073741824 //right shift by 2^30
4113
4114	if(ii<numToPrint)
4115	printf "%0.2b ",type
4116	endif
4117
4118	if(type == 0)
4119	num0 += 1
4120	numXYevents += 1
4121	endif
4122	if(type == 2)
4123	num2 += 1
4124	numXYevents += 1
4125	endif
4126	if(type == 1)
4127	num1 += 1
4128	endif
4129	if(type == 3)
4130	num3 += 1
4131	endif
4132
4133	bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
4134	if(ii<numToPrint)
4135	printf "%0.1b ",bit29
4136	endif
4137
4138	if(type==0 \|\| type==2)
4139	numPP += round(bit29)
4140	endif
4141
4142	if(type==1 \|\| type==3)
4143	numT0 += round(bit29)
4144	endif
4145
4146	if(dataval == 0)
4147	numZero += 1
4148	endif
4149
4150	if(ii<numToPrint)
4151	printf "\r"
4152	ii += 1
4153	endif
4154
4155	while(1)
4156	Close fileref
4157	toc()
4158
4159	Print "(Igor) numT0 = ",numT0
4160	Print "num0 = ",num0
4161	Print "num1 = ",num1
4162	Print "num2 = ",num2
4163	Print "num3 = rollover = ",num3
4164	Print "numXY = ",numXYevents
4165	Print "numZero = ",numZero
4166	Print "numPP = ",numPP
4167
4168	return(0)
4169
4170	End

Note: See TracBrowser for help on using the repository browser.

Download in other formats:

Original Format