Context Navigation

EventModeProcessing.ipf @ 916

Last change on this file since 916 was 916, checked in by srkline, 10 years ago

Changes to the panel sizes to accommodate smaller screens on laptops. Resized some panels and reorganized some of the controls. Help files have been updated to reflect these changes.

Added function to rename raw data files by copying file on disk, then writing in the new file number to the header. Solves the ICE problem of duplicated run numbers. In NCNR_DataReadWrite. See RenumberRunNumber?

Added new functions to the EventModeProcessing? to split large files (stream mode) and decimate when they are read back in. Added features to the event editing to automatically detect steps, and to show the differential with the data. Panel has improved layout. Still in the beta phase with these new features.

File size: 85.1 KB

Line
1	#pragma rtGlobals=1 // Use modern global access method.
2	#pragma IgorVersion=6.22
3
4	// vers 7.13e
5
6	// TODO:
7	//
8	// -- search for TODO for unresolved issues not on this list
9	//
10	// -- add comments to the code as needed
11	//
12	// -- write the help file, and link the help buttons to the help docs
13	//
14	// -- examples?
15	//
16	// x- add the XOP to the distribution package
17	//
18	// -- Need to make sure that the rescaledTime and the differentiated time graphs are
19	// being properly updated when the data is processed, modified, etc.
20	//
21	// -- I need better nomenclature other than "stream" for the "continuous" data set.
22	// It's all a stream, just sometimes it's not oscillatory
23	//
24	//
25	// -- the slice display "fails" for data sets that have 3 or 4 slices, as the ModifyImage command
26	// interprets the data as being RGB - and so does nothing.
27	// need to find a way around this
28	//
29	// -- Do something with the PP events. Currently, only the PP events that are XY (just the
30	// type 0 events (since I still need to find out what they realy mean)
31	//
32	// -- Add a switch to allow Sorting of the Stream data to remove the "time-reversed" data
33	// points. Maybe not kosher, but would clean things up.
34	//
35	//
36	///////// DONE //////////
37	//
38	// X- memory issues:
39	// -- in LoadEvents -- should I change the MAKE to:
40	// /I/U is unsigned 32-bit integer (for the time)
41	// /B/U is unsigned 8-bit integer (max val=255) for the x and y values
42	// -- then how does this affect downstream processing - such as rescaledTime, differentiation, etc.
43	// x- and can I re-write the XOP to create these types of data waves, and properly fill them...
44	//
45	// **- any integer waves must be translated by Igor into FP to be able to be displayed or for any
46	// type of analysis. so it's largely a waste of time to use integers. so simply force the XOP to
47	// generate only SP waves. this will at least save some space.
48	//
49	//
50	//
51	// x- Is there any way to improve the speed of the loader? How could an XOP be structured
52	// for maximum flexibility? Leave the post processing to Igor, but how much for the XOP
53	// to do? And can it handle such large amounts of data to pass back and forth, or
54	// does it need to be written as an operation, rather than a function??? I'd really
55	// rather that Igor handles the memory management, not me, if I write the XOP.
56	//
57	// **- as of 11/27, the OSX version of the XOP event loader is about 35x faster for the load!
58	// and is taking approx 1.8s/28MB, or about 6.5s/100MB of file. quite reasonable now, and
59	// probably a bit faster yet on the PC.
60	//
61	//
62	// X- fix the log/lin display - it's not working correctly
63	// I could use ModifyImage and log = 0\|1 keyword for the log Z display
64	// rather than creating a duplicate wave of log(data)
65	// -- it's in the Function sliceSelectEvent_Proc()
66	//
67	// X- add controls to show the bar graph
68	// x- add popup for selecting the binning type
69	// x- add ability to save the slices to RAW VAX files
70	// X- add control to show the bin counts and bin end times
71	// x- ADD buttons, switches, etc for the oscillatory mode - so that this can be accessed
72	//
73	// x- How are the headers filled for the VAX files from Teabag???
74	// -- I currently read the events 2x. Once to count the events to make the waves the proper
75	// size, then a second time to actualy process the events. Would it be faster to insert points
76	// as needed, or to estimate the size, and make it too large, then trim at the end...
77	// ((( NO -- I have no good way of getting a proper estimate of how many XY events there are for a file))
78	//
79	//
80	//
81
82
83	//
84	// These are also defined in the TISANE procedure file. In both files they are declared
85	// as Static, so they are local to each procedure
86	//
87	Static Constant ATXY = 0
88	Static Constant ATXYM = 2
89	Static Constant ATMIR = 1
90	Static Constant ATMAR = 3
91
92	Static Constant USECSPERTICK=0.1 // microseconds
93	Static Constant TICKSPERUSEC=10
94	Static Constant XBINS=128
95	Static Constant YBINS=128
96	//
97
98	Static Constant MODE_STREAM = 0
99	Static Constant MODE_OSCILL = 1
100	Static Constant MODE_TISANE = 2
101	Static Constant MODE_TOF = 3
102
103	//Menu "Macros"
104	// "Split Large File",SplitBigFile()
105	// "Accumulate First Slice",AccumulateSlices(0)
106	// "Add Current Slice",AccumulateSlices(1)
107	// "Display Accumulated Slices",AccumulateSlices(2)
108	//End
109
110
111
112	Proc Show_Event_Panel()
113	DoWindow/F EventModePanel
114	if(V_flag ==0)
115	Init_Event()
116	EventModePanel()
117	EndIf
118	End
119
120
121	Function Init_Event()
122	String/G root:Packages:NIST:gEvent_logfile
123	String/G root:Packages:NIST:gEventDisplayString="Details of the file load"
124
125	Variable/G root:Packages:NIST:AIMTYPE_XY=0 // XY Event
126	Variable/G root:Packages:NIST:AIMTYPE_XYM=2 // XY Minor event
127	Variable/G root:Packages:NIST:AIMTYPE_MIR=1 // Minor rollover event
128	Variable/G root:Packages:NIST:AIMTYPE_MAR=3 // Major rollover event
129
130	Variable/G root:Packages:NIST:gEvent_time_msw = 0
131	Variable/G root:Packages:NIST:gEvent_time_lsw = 0
132	Variable/G root:Packages:NIST:gEvent_t_longest = 0
133
134	Variable/G root:Packages:NIST:gEvent_tsdisp //Displayed slice
135	Variable/G root:Packages:NIST:gEvent_nslices = 10 //Number of time slices
136
137	Variable/G root:Packages:NIST:gEvent_logint = 1
138
139	Variable/G root:Packages:NIST:gEvent_Mode = MODE_OSCILL // ==0 for "stream", ==1 for Oscillatory
140	Variable/G root:Packages:NIST:gRemoveBadEvents = 1 // ==1 to remove "bad" events, ==0 to read "as-is"
141	Variable/G root:Packages:NIST:gSortStreamEvents = 0 // ==1 to sort the event stream, a last resort for a stream of data
142
143	Variable/G root:Packages:NIST:gEvent_ForceTmaxBin=1 //==1 to enforce t_longest in user-defined custom bins
144
145	NVAR nslices = root:Packages:NIST:gEvent_nslices
146
147	SetDataFolder root:
148	NewDataFolder/O/S root:Packages:NIST:Event
149
150	Make/D/O/N=(XBINS,YBINS,nslices) slicedData
151	Duplicate/O slicedData logslicedData
152	Duplicate/O slicedData dispsliceData
153
154
155	// for decimation
156	Variable/G root:Packages:NIST:Event:gDecimation = 100
157	Variable/G root:Packages:NIST:gEvent_t_longest_decimated = 0
158
159	// for large file splitting
160	String/G root:Packages:NIST:Event:gSplitFileList = "" // a list of the file names as split
161
162	SetDataFolder root:
163	End
164
165	//
166	// -- extra bits of buttons... not used
167	//
168	// Button button9 title="Decimation",size={100,20},pos={490,400},proc=E_ShowDecimateButton
169	//
170	// Button button11,pos={490,245},size={150,20},proc=LoadDecimateButtonProc,title="Load and Decimate"
171	// Button button12,pos={490,277},size={150,20},proc=ConcatenateButtonProc,title="Concatenate"
172	// Button button13,pos={490,305},size={150,20},proc=DisplayConcatenatedButtonProc,title="Display Concatenated"
173	//
174	// GroupBox group0 title="Manual Controls",size={185,112},pos={490,220}
175	//
176	// NewPanel /W=(82,44,854,664)/N=EventModePanel/K=2
177	// DoWindow/C EventModePanel
178	// ModifyPanel fixedSize=1,noEdit =1
179	Proc EventModePanel()
180	PauseUpdate; Silent 1 // building window...
181	NewPanel /W=(82,44,854,664)/N=EventModePanel/K=2
182	DoWindow/C EventModePanel
183	ModifyPanel fixedSize=1,noEdit =1
184
185	SetDrawLayer UserBack
186	DrawText 479,345,"Stream Data"
187	DrawLine 563,338,731,338
188	DrawText 479,419,"Oscillatory Data"
189	DrawLine 578,411,731,411
190
191	// ShowTools/A
192	Button button0,pos={14,87},size={150,20},proc=LoadEventLog_Button,title="Load Event Log File"
193	Button button0,fSize=12
194	TitleBox tb1,pos={475,500},size={135,24},fSize=10
195	TitleBox tb1,variable= root:Packages:NIST:gEventDisplayString
196
197	CheckBox chkbox2,pos={376,151},size={81,14},proc=LogIntEvent_Proc,title="Log Intensity"
198	CheckBox chkbox2,fSize=10,variable= root:Packages:NIST:gEvent_logint
199	CheckBox chkbox3,pos={14,125},size={119,14},title="Remove Bad Events?",fSize=10
200	CheckBox chkbox3,variable= root:Packages:NIST:gRemoveBadEvents
201
202	Button doneButton,pos={708,36},size={50,20},proc=EventDone_Proc,title="Done"
203	Button doneButton,fSize=12
204	Button button2,pos={419,28},size={140,20},proc=ShowEventDataButtonProc,title="Show Event Data"
205	Button button3,pos={419,56},size={140,20},proc=ShowBinDetailsButtonProc,title="Show Bin Details"
206	Button button4,pos={487,227},size={120,20},proc=UndoTimeSortButtonProc,title="Undo Time Sort"
207	Button button5,pos={419,85},size={140,20},proc=ExportSlicesButtonProc,title="Export Slices as VAX"
208	Button button6,pos={718,9},size={40,20},proc=EventModeHelpButtonProc,title="?"
209	Button button7,pos={487,197},size={120,20},proc=AdjustEventDataButtonProc,title="Adjust Events"
210	Button button8,pos={619,197},size={120,20},proc=CustomBinButtonProc,title="Custom Bins"
211	Button button1,pos={206,100},size={120,20},proc=ProcessEventLog_Button,title="Bin Event Data"
212	Button button1,fSize=12
213
214	SetVariable setvar0,pos={208,149},size={160,16},proc=sliceSelectEvent_Proc,title="Display Time Slice"
215	SetVariable setvar0,fSize=10
216	SetVariable setvar0,limits={0,1000,1},value= root:Packages:NIST:gEvent_tsdisp
217	SetVariable setvar1,pos={206,26},size={160,16},title="Number of slices",fSize=10
218	SetVariable setvar1,limits={1,1000,1},value= root:Packages:NIST:gEvent_nslices
219	SetVariable setvar2,pos={206,51},size={160,16},title="Max Time (s)",fSize=10
220	SetVariable setvar2,value= root:Packages:NIST:gEvent_t_longest
221
222	PopupMenu popup0,pos={206,74},size={119,20},proc=BinTypePopMenuProc,title="Bin Spacing"
223	PopupMenu popup0,fSize=10
224	PopupMenu popup0,mode=1,popvalue="Equal",value= #"\"Equal;Fibonacci;Custom;\""
225
226	Button button10,pos={488,305},size={100,20},proc=SplitFileButtonProc,title="Split Big File"
227	Button button14,pos={488,350},size={130,20},proc=Stream_LoadDecim,title="Load From List"
228	SetVariable setvar3,pos={487,378},size={150,16},title="Decimation factor"
229	SetVariable setvar3,fSize=10
230	SetVariable setvar3,limits={1,inf,1},value= root:Packages:NIST:Event:gDecimation
231
232	Button button15_0,pos={488,425},size={110,20},proc=AccumulateSlicesButton,title="Add First Slice"
233	Button button16_1,pos={488,450},size={110,20},proc=AccumulateSlicesButton,title="Add Next Slice"
234	Button button17_2,pos={620,425},size={110,20},proc=AccumulateSlicesButton,title="Display Total"
235
236
237	CheckBox chkbox1_0,pos={25,34},size={69,14},title="Oscillatory",fSize=10
238	CheckBox chkbox1_0,mode=1,proc=EventModeRadioProc,value=1
239	CheckBox chkbox1_1,pos={25,59},size={53,14},title="Stream",fSize=10
240	CheckBox chkbox1_1,proc=EventModeRadioProc,value=0,mode=1
241	CheckBox chkbox1_2,pos={104,59},size={53,14},title="TISANE",fSize=10
242	CheckBox chkbox1_2,proc=EventModeRadioProc,value=0,mode=1
243	CheckBox chkbox1_3,pos={104,34},size={37,14},title="TOF",fSize=10
244	CheckBox chkbox1_3,proc=EventModeRadioProc,value=0,mode=1
245
246	GroupBox group0_0,pos={5,5},size={174,112},title="Mode",fSize=12,fStyle=1
247	GroupBox group0_1,pos={192,5},size={192,123},title="Binning",fSize=12,fStyle=1
248	GroupBox group0_2,pos={403,5},size={173,109},title="Viewing",fSize=12,fStyle=1
249	GroupBox group0_3,pos={476,169},size={277,92},title="Editing",fSize=12,fStyle=1
250	GroupBox group0_4,pos={474,278},size={279,200},title="Big Files",fSize=12
251	GroupBox group0_4,fStyle=1
252
253	Display/W=(10,170,460,610)/HOST=#
254	AppendImage/T :Packages:NIST:Event:dispsliceData
255	ModifyImage dispsliceData ctab= {,,ColdWarm,0}
256	ModifyImage dispsliceData ctabAutoscale=3
257	ModifyGraph margin(left)=14,margin(bottom)=14,margin(top)=14,margin(right)=14
258	ModifyGraph mirror=2
259	ModifyGraph nticks=4
260	ModifyGraph minor=1
261	ModifyGraph fSize=9
262	ModifyGraph standoff=0
263	ModifyGraph tkLblRot(left)=90
264	ModifyGraph btLen=3
265	ModifyGraph tlOffset=-2
266	RenameWindow #,Event_slicegraph
267	SetActiveSubwindow ##
268	EndMacro
269
270
271
272
273	// mode selector
274	//Static Constant MODE_STREAM = 0
275	//Static Constant MODE_OSCILL = 1
276	//Static Constant MODE_TISANE = 2
277	//Static Constant MODE_TOF = 3
278	//
279	Function EventModeRadioProc(name,value)
280	String name
281	Variable value
282
283	NVAR gEventModeRadioVal= root:Packages:NIST:gEvent_mode
284
285	strswitch (name)
286	case "chkbox1_0":
287	gEventModeRadioVal= MODE_OSCILL
288	break
289	case "chkbox1_1":
290	gEventModeRadioVal= MODE_STREAM
291	break
292	case "chkbox1_2":
293	gEventModeRadioVal= MODE_TISANE
294	break
295	case "chkbox1_3":
296	gEventModeRadioVal= MODE_TOF
297	break
298	endswitch
299	CheckBox chkbox1_0,value= gEventModeRadioVal==MODE_OSCILL
300	CheckBox chkbox1_1,value= gEventModeRadioVal==MODE_STREAM
301	CheckBox chkbox1_2,value= gEventModeRadioVal==MODE_TISANE
302	CheckBox chkbox1_3,value= gEventModeRadioVal==MODE_TOF
303
304	return(0)
305	End
306
307	Function AdjustEventDataButtonProc(ba) : ButtonControl
308	STRUCT WMButtonAction &ba
309
310	switch( ba.eventCode )
311	case 2: // mouse up
312	// click code here
313	Execute "ShowEventCorrectionPanel()"
314	//
315	break
316	case -1: // control being killed
317	break
318	endswitch
319
320	return 0
321	End
322
323	Function CustomBinButtonProc(ba) : ButtonControl
324	STRUCT WMButtonAction &ba
325
326	switch( ba.eventCode )
327	case 2: // mouse up
328	// click code here
329	Execute "Show_CustomBinPanel()"
330	//
331	break
332	case -1: // control being killed
333	break
334	endswitch
335
336	return 0
337	End
338
339
340	Function ShowEventDataButtonProc(ba) : ButtonControl
341	STRUCT WMButtonAction &ba
342
343	switch( ba.eventCode )
344	case 2: // mouse up
345	// click code here
346	Execute "ShowRescaledTimeGraph()"
347	//
348	DifferentiatedTime()
349	//
350	break
351	case -1: // control being killed
352	break
353	endswitch
354
355	return 0
356	End
357
358	Function BinTypePopMenuProc(pa) : PopupMenuControl
359	STRUCT WMPopupAction &pa
360
361	switch( pa.eventCode )
362	case 2: // mouse up
363	Variable popNum = pa.popNum
364	String popStr = pa.popStr
365	if(cmpstr(popStr,"Custom")==0)
366	Execute "Show_CustomBinPanel()"
367	endif
368	break
369	case -1: // control being killed
370	break
371	endswitch
372
373	return 0
374	End
375
376	Function ShowBinDetailsButtonProc(ba) : ButtonControl
377	STRUCT WMButtonAction &ba
378
379	switch( ba.eventCode )
380	case 2: // mouse up
381	// click code here
382	Execute "ShowBinTable()"
383	Execute "BinEventBarGraph()"
384	break
385	case -1: // control being killed
386	break
387	endswitch
388
389	return 0
390	End
391
392	Function UndoTimeSortButtonProc(ba) : ButtonControl
393	STRUCT WMButtonAction &ba
394
395	switch( ba.eventCode )
396	case 2: // mouse up
397	// click code here
398	Execute "UndoTheSorting()"
399	break
400	case -1: // control being killed
401	break
402	endswitch
403
404	return 0
405	End
406
407	Function ExportSlicesButtonProc(ba) : ButtonControl
408	STRUCT WMButtonAction &ba
409
410	switch( ba.eventCode )
411	case 2: // mouse up
412	// click code here
413	Execute "ExportSlicesAsVAX()" //will invoke the dialog
414	break
415	case -1: // control being killed
416	break
417	endswitch
418
419	return 0
420	End
421
422	Function EventModeHelpButtonProc(ba) : ButtonControl
423	STRUCT WMButtonAction &ba
424
425	switch( ba.eventCode )
426	case 2: // mouse up
427	// click code here
428	DisplayHelpTopic/Z "Event Mode Data"
429	break
430	case -1: // control being killed
431	break
432	endswitch
433
434	return 0
435	End
436
437
438	Function EventDone_Proc(ba) : ButtonControl
439	STRUCT WMButtonAction &ba
440
441	String win = ba.win
442	switch (ba.eventCode)
443	case 2:
444	DoWindow/K EventModePanel
445	break
446	endswitch
447	return(0)
448	End
449
450
451
452	Function ProcessEventLog_Button(ctrlName) : ButtonControl
453	String ctrlName
454
455	NVAR mode=root:Packages:NIST:gEvent_Mode
456
457	if(mode == MODE_STREAM)
458	Stream_ProcessEventLog("")
459	endif
460
461	if(mode == MODE_OSCILL)
462	Osc_ProcessEventLog("")
463	endif
464
465	// toggle the checkbox for log display to force the display to be correct
466	NVAR gLog = root:Packages:NIST:gEvent_logint
467	LogIntEvent_Proc("",gLog)
468
469	return(0)
470	end
471
472	// for oscillatory mode
473	//
474	Function Osc_ProcessEventLog(ctrlName)
475	String ctrlName
476
477	Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
478
479	Wave binnedData = root:Packages:NIST:Event:binnedData
480	Wave xLoc = root:Packages:NIST:Event:xLoc
481	Wave yLoc = root:Packages:NIST:Event:yLoc
482
483	// now with the number of slices and max time, process the events
484
485	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
486	NVAR nslices = root:Packages:NIST:gEvent_nslices
487
488	SetDataFolder root:Packages:NIST:Event //don't count on the folder remaining here
489
490	Make/D/O/N=(XBINS,YBINS,nslices) slicedData
491
492	Wave slicedData = slicedData
493	Wave rescaledTime = rescaledTime
494	Wave timePt = timePt
495	Make/O/D/N=(128,128) tmpData
496	Make/O/D/N=(nslices+1) binEndTime,binCount
497	Make/O/D/N=(nslices) timeWidth
498	Wave timeWidth = timeWidth
499	Wave binEndTime = binEndTime
500	Wave binCount = binCount
501
502	variable ii,del,p1,p2,t1,t2
503	del = t_longest/nslices
504
505	slicedData = 0
506	binEndTime[0]=0
507	BinCount[nslices]=0
508
509
510	String binTypeStr=""
511	ControlInfo /W=EventModePanel popup0
512	binTypeStr = S_value
513
514	strswitch(binTypeStr) // string switch
515	case "Equal": // execute if case matches expression
516	SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
517	break // exit from switch
518	case "Fibonacci": // execute if case matches expression
519	SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
520	break
521	case "Log": // execute if case matches expression
522	SetLogBins(binEndTime,timeWidth,nslices,t_longest)
523	break
524	case "Custom": // execute if case matches expression
525	//bins are set by the user on the panel - assume it's good to go
526	break
527	default: // optional default expression executed
528	DoAlert 0,"No match for bin type, Equal bins used"
529	SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
530	endswitch
531
532
533	// now before binning, sort the data
534
535	//this is slow - undoing the sorting and starting over, but if you don't,
536	// you'll never be able to undo the sort
537	//
538	SetDataFolder root:Packages:NIST:Event:
539
540	if(WaveExists($"root:Packages:NIST:Event:OscSortIndex") == 0 )
541	Duplicate/O rescaledTime OscSortIndex
542	MakeIndex rescaledTime OscSortIndex
543	IndexSort OscSortIndex, yLoc,xLoc,timePt,rescaledTime
544	//SetDataFolder root:Packages:NIST:Event
545	IndexForHistogram(xLoc,yLoc,binnedData) // index the events AFTER sorting
546	//SetDataFolder root:
547	Endif
548
549	Wave index = root:Packages:NIST:Event:SavedIndex //this is the histogram index
550
551	for(ii=0;ii<nslices;ii+=1)
552	if(ii==0)
553	// t1 = ii*del
554	// t2 = (ii+1)*del
555	p1 = BinarySearch(rescaledTime,0)
556	p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
557	else
558	// t2 = (ii+1)*del
559	p1 = p2+1 //one more than the old one
560	p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
561	endif
562
563	// typically zero will never be a valid time value in oscillatory mode. in "stream" mode, the first is normalized to == 0
564	// but not here - times are what they are.
565	if(p1 == -1)
566	Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
567	p1 = 0 //set to the first point if it's off the end
568	Endif
569
570	if(p2 == -2)
571	Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
572	p2 = numpnts(rescaledTime)-1 //set to the last point if it's off the end
573	Endif
574	// Print p1,p2
575
576
577	tmpData=0
578	JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
579	slicedData[][][ii] = tmpData[p][q]
580
581	// binEndTime[ii+1] = t2
582	binCount[ii] = sum(tmpData,-inf,inf)
583	endfor
584
585	Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
586	Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
587	logslicedData = log(slicedData)
588
589	SetDataFolder root:
590	return(0)
591	End
592
593	// for a "continuous exposure"
594	//
595	// if there is a sort of these events, I need to re-index the events for the histogram
596	// - see the oscillatory mode - and sort the events here, then immediately re-index for the histogram
597	// - but with the added complication that I need to always remember to index for the histogram, every time
598	// - since I don't know if I've sorted or un-sorted. Osc mode always forces a re-sort and a re-index
599	//
600	Function Stream_ProcessEventLog(ctrlName)
601	String ctrlName
602
603	// NVAR slicewidth = root:Packages:NIST:gTISANE_slicewidth
604
605
606	Make/O/D/N=(128,128) root:Packages:NIST:Event:binnedData
607
608	Wave binnedData = root:Packages:NIST:Event:binnedData
609	Wave xLoc = root:Packages:NIST:Event:xLoc
610	Wave yLoc = root:Packages:NIST:Event:yLoc
611
612	// now with the number of slices and max time, process the events
613
614	NVAR yesSortStream = root:Packages:NIST:gSortStreamEvents //do I sort the events?
615	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
616	NVAR nslices = root:Packages:NIST:gEvent_nslices
617
618	SetDataFolder root:Packages:NIST:Event //don't count on the folder remaining here
619
620	Make/D/O/N=(XBINS,YBINS,nslices) slicedData
621
622	Wave slicedData = slicedData
623	Wave rescaledTime = rescaledTime
624	Make/O/D/N=(128,128) tmpData
625	Make/O/D/N=(nslices+1) binEndTime,binCount//,binStartTime
626	Make/O/D/N=(nslices) timeWidth
627	Wave binEndTime = binEndTime
628	Wave timeWidth = timeWidth
629	Wave binCount = binCount
630
631	variable ii,del,p1,p2,t1,t2
632	del = t_longest/nslices
633
634	slicedData = 0
635	binEndTime[0]=0
636	BinCount[nslices]=0
637
638	String binTypeStr=""
639	ControlInfo /W=EventModePanel popup0
640	binTypeStr = S_value
641
642	strswitch(binTypeStr) // string switch
643	case "Equal": // execute if case matches expression
644	SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
645	break // exit from switch
646	case "Fibonacci": // execute if case matches expression
647	SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
648	break
649	case "Log": // execute if case matches expression
650	SetLogBins(binEndTime,timeWidth,nslices,t_longest)
651	break
652	case "Custom": // execute if case matches expression
653	//bins are set by the user on the panel - assume it's good to go
654	break
655	default: // optional default expression executed
656	DoAlert 0,"No match for bin type, Equal bins used"
657	SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
658	endswitch
659
660	// TODO
661	// the global exists for this switch, but it is not implemented - not sure whether
662	// it's correct to implement this at all --
663	//
664	if(yesSortStream == 1)
665	SortTimeData()
666	endif
667
668	// index the events before binning
669	// if there is a sort of these events, I need to re-index the events for the histogram
670	// SetDataFolder root:Packages:NIST:Event
671	IndexForHistogram(xLoc,yLoc,binnedData)
672	// SetDataFolder root:
673	Wave index = root:Packages:NIST:Event:SavedIndex //the index for the histogram
674
675
676	for(ii=0;ii<nslices;ii+=1)
677	if(ii==0)
678	// t1 = ii*del
679	// t2 = (ii+1)*del
680	p1 = BinarySearch(rescaledTime,0)
681	p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
682	else
683	// t2 = (ii+1)*del
684	p1 = p2+1 //one more than the old one
685	p2 = BinarySearch(rescaledTime,binEndTime[ii+1])
686	endif
687
688	if(p1 == -1)
689	Printf "p1 = -1 Binary search off the end %15.10g <?? %15.10g\r", 0, rescaledTime[0]
690	p1 = 0 //set to the first point if it's off the end
691	Endif
692	if(p2 == -2)
693	Printf "p2 = -2 Binary search off the end %15.10g >?? %15.10g\r", binEndTime[ii+1], rescaledTime[numpnts(rescaledTime)-1]
694	p2 = numpnts(rescaledTime)-1 //set to the last point if it's off the end
695	Endif
696	// Print p1,p2
697
698
699	tmpData=0
700	JointHistogramWithRange(xLoc,yLoc,tmpData,index,p1,p2)
701	slicedData[][][ii] = tmpData[p][q]
702
703	// binEndTime[ii+1] = t2
704	binCount[ii] = sum(tmpData,-inf,inf)
705	endfor
706
707	Duplicate/O slicedData,root:Packages:NIST:Event:dispsliceData,root:Packages:NIST:Event:logSlicedData
708	Wave logSlicedData = root:Packages:NIST:Event:logSlicedData
709	logslicedData = log(slicedData)
710
711	SetDataFolder root:
712	return(0)
713	End
714
715
716	Proc UndoTheSorting()
717	Osc_UndoSort()
718	End
719
720	// for oscillatory mode
721	//
722	// -- this takes the previously generated index, and un-sorts the data to restore to the
723	// "as-collected" state
724	//
725	Function Osc_UndoSort()
726
727	SetDataFolder root:Packages:NIST:Event //don't count on the folder remaining here
728	Wave rescaledTime = rescaledTime
729	Wave OscSortIndex = OscSortIndex
730	Wave yLoc = yLoc
731	Wave xLoc = xLoc
732	Wave timePt = timePt
733
734	Sort OscSortIndex OscSortIndex,yLoc,xLoc,timePt,rescaledTime
735
736	KillWaves/Z OscSortIndex
737
738	SetDataFolder root:
739	return(0)
740	End
741
742
743	// now before binning, sort the data
744	//
745	//this is slow - undoing the sorting and starting over, but if you don't,
746	// you'll never be able to undo the sort
747	//
748	Function SortTimeData()
749
750
751	SetDataFolder root:Packages:NIST:Event:
752
753	KillWaves/Z OscSortIndex
754
755	if(WaveExists($"root:Packages:NIST:Event:OscSortIndex") == 0 )
756	Duplicate/O rescaledTime OscSortIndex
757	MakeIndex rescaledTime OscSortIndex
758	IndexSort OscSortIndex, yLoc,xLoc,timePt,rescaledTime
759	Endif
760
761	SetDataFolder root:
762	return(0)
763	End
764
765
766
767	Function SetLinearBins(binEndTime,timeWidth,nslices,t_longest)
768	Wave binEndTime,timeWidth
769	Variable nslices,t_longest
770
771	Variable del,ii,t2
772	binEndTime[0]=0 //so the bar graph plots right...
773	del = t_longest/nslices
774
775	for(ii=0;ii<nslices;ii+=1)
776	t2 = (ii+1)*del
777	binEndTime[ii+1] = t2
778	endfor
779	binEndTime[ii+1] = t_longest*(1-1e-6) //otherwise floating point errors such that the last time point is off the end of the Binary search
780
781	timeWidth = binEndTime[p+1]-binEndTime[p]
782
783	return(0)
784	End
785
786	// TODO
787	// either get this to work, or scrap it entirely. it currently isn't on the popup
788	// so it can't be accessed
789	Function SetLogBins(binEndTime,timeWidth,nslices,t_longest)
790	Wave binEndTime,timeWidth
791	Variable nslices,t_longest
792
793	Variable tMin,ii
794
795	Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
796
797	binEndTime[0]=0 //so the bar graph plots right...
798
799	// just like the log-scaled q-points
800	tMin = rescaledTime[1]/1 //just a guess... can't use tMin=0, and rescaledTime[0] == 0 by definition
801	Print rescaledTime[1], tMin
802	for(ii=0;ii<nslices;ii+=1)
803	binEndTime[ii+1] =alog(log(tMin) + (ii+1)*((log(t_longest)-log(tMin))/nslices))
804	endfor
805	binEndTime[ii+1] = t_longest //otherwise floating point errors such that the last time point is off the end of the Binary search
806
807	timeWidth = binEndTime[p+1]-binEndTime[p]
808
809	return(0)
810	End
811
812	Function MakeFibonacciWave(w,num)
813	Wave w
814	Variable num
815
816	//skip the initial zero
817	Variable f1,f2,ii
818	f1=1
819	f2=1
820	w[0] = f1
821	w[1] = f2
822	for(ii=2;ii<num;ii+=1)
823	w[ii] = f1+f2
824	f1=f2
825	f2=w[ii]
826	endfor
827
828	return(0)
829	end
830
831	Function SetFibonacciBins(binEndTime,timeWidth,nslices,t_longest)
832	Wave binEndTime,timeWidth
833	Variable nslices,t_longest
834
835	Variable tMin,ii,total,t2,tmp
836	Make/O/D/N=(nslices) fibo
837	fibo=0
838	MakeFibonacciWave(fibo,nslices)
839
840	// Make/O/D tmpFib={1,1,2,3,5,8,13,21,34,55,89,144,233,377,610,987,1597,2584,4181,6765,10946}
841
842	binEndTime[0]=0 //so the bar graph plots right...
843	total = sum(fibo,0,nslices-1) //total number of "pieces"
844
845	tmp=0
846	for(ii=0;ii<nslices;ii+=1)
847	t2 = sum(fibo,0,ii)/total*t_longest
848	binEndTime[ii+1] = t2
849	endfor
850	binEndTime[ii+1] = t_longest //otherwise floating point errors such that the last time point is off the end of the Binary search
851
852	timeWidth = binEndTime[p+1]-binEndTime[p]
853
854	return(0)
855	End
856
857
858
859	// TODO:
860	//
861	// What, if anything is different about the OSC or STREAM load?
862	// I think that only the processing is different. so this could be
863	// consolidated into a single loader.
864	//
865	// ** currently, the "stream" loader uses the first data point as time=0
866	// and rescales everything to that time. "Osc" loading uses the times "as-is"
867	// from the file, trusting the times to be correct.
868	//
869	// Would TISANE or TOF need a different loader?
870	//
871	Function LoadEventLog_Button(ctrlName) : ButtonControl
872	String ctrlName
873
874	NVAR mode=root:Packages:NIST:gEvent_mode
875	Variable err=0
876	Variable fileref,totBytes
877	Variable fileTooLarge = 150 //limit load to 150MB
878
879	SVAR filename = root:Packages:NIST:gEvent_logfile
880	NVAR nslices = root:Packages:NIST:gEvent_nslices
881	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
882
883	String fileFilters = "All Files:.;Data Files (.txt):.txt;"
884	String abortStr
885
886	Open/R/D/F=fileFilters fileref
887	filename = S_filename
888	if(strlen(S_filename) == 0)
889	// user cancelled
890	DoAlert 0,"No file selected, no file loaded."
891	return(1)
892	endif
893
894	/// Abort if the files are too large
895	Open/R fileref as fileName
896	FStatus fileref
897	Close fileref
898
899	totBytes = V_logEOF/1e6 //in MB
900	if(totBytes > fileTooLarge)
901	sprintf abortStr,"File is %g MB, larger than the limit of %g MB. Split and Decimate.",totBytes,fileTooLarge
902	Abort abortStr
903	endif
904
905
906	#if (exists("EventLoadWave")==4)
907	LoadEvents_XOP()
908	#else
909	LoadEvents()
910	#endif
911
912	SetDataFolder root:Packages:NIST:Event:
913
914	//tic()
915	Wave timePt=timePt
916	Wave xLoc=xLoc
917	Wave yLoc=yLoc
918	CleanupTimes(xLoc,yLoc,timePt) //remove zeroes
919	//toc()
920
921
922	/////
923	// now do a little processing of the times based on the type of data
924	//
925	if(mode == MODE_STREAM) // continuous "Stream" mode - start from zero
926	Duplicate/O timePt rescaledTime
927	rescaledTime = 1e-7*(timePt-timePt[0]) //convert to seconds and start from zero
928	t_longest = waveMax(rescaledTime) //should be the last point
929	endif
930
931	if(mode == MODE_OSCILL) // oscillatory mode - don't adjust the times, we get periodic t0 to reset t=0
932	Duplicate/O timePt rescaledTime
933	rescaledTime *= 1e-7 //convert to seconds and that's all
934	t_longest = waveMax(rescaledTime) //if oscillatory, won't be the last point, so get it this way
935
936	KillWaves/Z OscSortIndex //to make sure that there is no old index hanging around
937	endif
938
939
940
941	SetDataFolder root:
942
943	STRUCT WMButtonAction ba
944	ba.eventCode = 2
945	ShowEventDataButtonProc(ba)
946
947	return(0)
948	End
949
950	//// for the mode of "one continuous exposure"
951	////
952	//Function Stream_LoadEventLog(ctrlName)
953	// String ctrlName
954	//
955	// Variable fileref
956	//
957	// SVAR filename = root:Packages:NIST:gEvent_logfile
958	// NVAR nslices = root:Packages:NIST:gEvent_nslices
959	// NVAR t_longest = root:Packages:NIST:gEvent_t_longest
960	//
961	// String fileFilters = "All Files:.;Data Files (.txt):.txt;"
962	//
963	// Open/R/D/F=fileFilters fileref
964	// filename = S_filename
965	// if(strlen(S_filename) == 0)
966	// // user cancelled
967	// DoAlert 0,"No file selected, no file loaded."
968	// return(1)
969	// endif
970	//
971	//#if (exists("EventLoadWave")==4)
972	// LoadEvents_XOP()
973	//#else
974	// LoadEvents()
975	//#endif
976	//
977	// SetDataFolder root:Packages:NIST:Event:
978	//
979	////tic()
980	// Wave timePt=timePt
981	// Wave xLoc=xLoc
982	// Wave yLoc=yLoc
983	// CleanupTimes(xLoc,yLoc,timePt) //remove zeroes
984	//
985	////toc()
986	//
987	// Duplicate/O timePt rescaledTime
988	// rescaledTime = 1e-7*(timePt-timePt[0]) //convert to seconds and start from zero
989	// t_longest = waveMax(rescaledTime) //should be the last point
990	//
991	// SetDataFolder root:
992	//
993	// return(0)
994	//End
995	//
996	//// for the mode "oscillatory"
997	////
998	//Function Osc_LoadEventLog(ctrlName)
999	// String ctrlName
1000	//
1001	// Variable fileref
1002	//
1003	// SVAR filename = root:Packages:NIST:gEvent_logfile
1004	// NVAR nslices = root:Packages:NIST:gEvent_nslices
1005	// NVAR t_longest = root:Packages:NIST:gEvent_t_longest
1006	//
1007	// String fileFilters = "All Files:.;Data Files (.txt):.txt;"
1008	//
1009	// Open/R/D/F=fileFilters fileref
1010	// filename = S_filename
1011	// if(strlen(S_filename) == 0)
1012	// // user cancelled
1013	// DoAlert 0,"No file selected, no file loaded."
1014	// return(1)
1015	// endif
1016	//
1017	//#if (exists("EventLoadWave")==4)
1018	// LoadEvents_XOP()
1019	//#else
1020	// LoadEvents()
1021	//#endif
1022	//
1023	// SetDataFolder root:Packages:NIST:Event:
1024	//
1025	// Wave timePt=timePt
1026	// Wave xLoc=xLoc
1027	// Wave yLoc=yLoc
1028	// CleanupTimes(xLoc,yLoc,timePt) //remove zeroes
1029	//
1030	// Duplicate/O timePt rescaledTime
1031	// rescaledTime *= 1e-7 //convert to seconds and that's all
1032	// t_longest = waveMax(rescaledTime) //if oscillatory, won't be the last point, so get it this way
1033	//
1034	// KillWaves/Z OscSortIndex //to make sure that there is no old index hanging around
1035	//
1036	// SetDataFolder root:
1037	//
1038	// return(0)
1039	//End
1040
1041
1042	//
1043	// -- MUCH faster to count the number of lines to remove, then delete (N)
1044	// rather then delete them one-by-one in the do-loop
1045	Function CleanupTimes(xLoc,yLoc,timePt)
1046	Wave xLoc,yLoc,timePt
1047
1048	// start at the back and remove zeros
1049	Variable num=numpnts(xLoc),ii,numToRemove
1050
1051	numToRemove = 0
1052	ii=num
1053	do
1054	ii -= 1
1055	if(timePt[ii] == 0 && xLoc[ii] == 0 && yLoc[ii] == 0)
1056	numToRemove += 1
1057	endif
1058	while(timePt[ii-1] == 0 && xLoc[ii-1] == 0 && yLoc[ii-1] == 0)
1059
1060	if(numToRemove != 0)
1061	DeletePoints ii, numToRemove, xLoc,yLoc,timePt
1062	endif
1063
1064	return(0)
1065	End
1066
1067	Function LogIntEvent_Proc(ctrlName,checked) : CheckBoxControl
1068	String ctrlName
1069	Variable checked
1070
1071	SetDataFolder root:Packages:NIST:Event
1072	if(checked)
1073	Duplicate/O logslicedData dispsliceData
1074	else
1075	Duplicate/O slicedData dispsliceData
1076	endif
1077
1078	NVAR selectedslice = root:Packages:NIST:gEvent_tsdisp
1079
1080	sliceSelectEvent_Proc("", selectedslice, "", "")
1081
1082	SetDataFolder root:
1083
1084	End
1085
1086
1087	// TODO
1088	// this "fails" for data sets that have 3 or 4 slices, as the ModifyImage command
1089	// interprets the data as being RGB - and so does nothing.
1090	// need to find a way around this
1091	///
1092	// I could modify this procedure to use the log = 0\|1 keyword for the log Z display
1093	// rather than creating a duplicate wave of log(data)
1094	//
1095	Function sliceSelectEvent_Proc(ctrlName, varNum, varStr, varName) : SetVariableControl
1096	String ctrlName
1097	Variable varNum
1098	String varStr
1099	String varName
1100
1101	NVAR nslices = root:Packages:NIST:gEvent_nslices
1102	NVAR selectedslice = root:Packages:NIST:gEvent_tsdisp
1103
1104	if(varNum < 0)
1105	selectedslice = 0
1106	DoUpdate
1107	elseif (varNum > nslices-1)
1108	selectedslice = nslices-1
1109	DoUpdate
1110	else
1111	ModifyImage/W=EventModePanel#Event_slicegraph ''#0 plane = varNum
1112	endif
1113
1114	End
1115
1116	Function DifferentiatedTime()
1117
1118	Wave rescaledTime = root:Packages:NIST:Event:rescaledTime
1119
1120	SetDataFolder root:Packages:NIST:Event:
1121
1122	Differentiate rescaledTime/D=rescaledTime_DIF
1123	// Display rescaledTime,rescaledTime_DIF
1124	DoWindow/F Differentiated_Time
1125	if(V_flag == 0)
1126	Display/N=Differentiated_Time/K=1 rescaledTime_DIF
1127	Legend
1128	Modifygraph gaps=0
1129	ModifyGraph zero(left)=1
1130	Label left "\\Z14Delta (dt/event)"
1131	Label bottom "\\Z14Event number"
1132	endif
1133
1134	SetDataFolder root:
1135
1136	return(0)
1137	End
1138
1139
1140	//
1141	// for the bit shifts, see the decimal-binary conversion
1142	// http://www.binaryconvert.com/convert_unsigned_int.html
1143	//
1144	// K0 = 536870912
1145	// Print (K0 & 0x08000000)/134217728 //bit 27 only, shift by 2^27
1146	// Print (K0 & 0x10000000)/268435456 //bit 28 only, shift by 2^28
1147	// Print (K0 & 0x20000000)/536870912 //bit 29 only, shift by 2^29
1148	//
1149	// This is duplicated by the XOP, but the Igor code allows quick access to print out
1150	// all of the gorey details of the events and every little bit of them. the print
1151	// statements and flags are kept for this reason, so the code is a bit messy.
1152	//
1153	Function LoadEvents()
1154
1155	NVAR time_msw = root:Packages:NIST:gEvent_time_msw
1156	NVAR time_lsw = root:Packages:NIST:gEvent_time_lsw
1157	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
1158
1159	SVAR filepathstr = root:Packages:NIST:gEvent_logfile
1160	SVAR dispStr = root:Packages:NIST:gEventDisplayString
1161
1162
1163	//// Variable decFac = 10 //decimation factor
1164	//// Variable jj,keep
1165
1166	SetDataFolder root:Packages:NIST:Event
1167
1168	Variable fileref
1169	String buffer
1170	String fileStr,tmpStr
1171	Variable dataval,timeval,type,numLines,verbose,verbose3
1172	Variable xval,yval,rollBit,nRoll,roll_time,bit29,bit28,bit27
1173	Variable ii,flaggedEvent,rolloverHappened,numBad=0,tmpPP=0,tmpT0=0
1174	Variable Xmax, yMax
1175
1176	xMax = 127 // number the detector from 0->127
1177	yMax = 127
1178
1179	verbose3 = 0 //prints out the rollover events (type==3)
1180	verbose = 0
1181	numLines = 0
1182
1183
1184	// what I really need is the number of XY events
1185	Variable numXYevents,num1,num2,num3,num0,totBytes,numPP,numT0,numDL,numFF,numZero
1186	Variable numRemoved
1187	numXYevents = 0
1188	num0 = 0
1189	num1 = 0
1190	num2 = 0
1191	num3 = 0
1192	numPP = 0
1193	numT0 = 0
1194	numDL = 0
1195	numFF = 0
1196	numZero = 0
1197	numRemoved = 0
1198
1199	//tic()
1200	Open/R fileref as filepathstr
1201	FStatus fileref
1202	Close fileref
1203
1204	totBytes = V_logEOF
1205	Print "total bytes = ", totBytes
1206
1207	//toc()
1208	//
1209
1210
1211	// do a "pre-scan to get some of the counts, so that I can allocate space. This does
1212	// double the read time, but is still faster than adding points to waves as the file is read
1213	//
1214
1215	tic()
1216
1217	Open/R fileref as filepathstr
1218	do
1219	do
1220	FReadLine fileref, buffer //skip the "blank" lines that have one character
1221	while(strlen(buffer) == 1)
1222
1223	if (strlen(buffer) == 0)
1224	break
1225	endif
1226
1227	sscanf buffer,"%x",dataval
1228
1229	// two most sig bits (31-30)
1230	type = (dataval & 0xC0000000)/1073741824 //right shift by 2^30
1231
1232	if(type == 0)
1233	num0 += 1
1234	numXYevents += 1
1235	endif
1236	if(type == 2)
1237	num2 += 1
1238	numXYevents += 1
1239	endif
1240	if(type == 1)
1241	num1 += 1
1242	endif
1243	if(type == 3)
1244	num3 += 1
1245	endif
1246
1247	bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1248
1249	if(type==0 \|\| type==2)
1250	numPP += round(bit29)
1251	endif
1252
1253	if(type==1 \|\| type==3)
1254	numT0 += round(bit29)
1255	endif
1256
1257	if(dataval == 0)
1258	numZero += 1
1259	endif
1260
1261	while(1)
1262	Close fileref
1263	// done counting the number of XY events
1264	printf("Igor pre-scan done in ")
1265	toc()
1266
1267
1268	Print "(Igor) numT0 = ",numT0
1269	Print "num0 = ",num0
1270	Print "num1 = ",num1
1271	Print "num2 = ",num2
1272	Print "num3 = ",num3
1273
1274	//
1275	//
1276	// Printf "numXYevents = %d\r",numXYevents
1277	// Printf "XY = num0 = %d\r",num0
1278	// Printf "XY time = num2 = %d\r",num2
1279	// Printf "time MSW = num1 = %d\r",num1
1280	// Printf "Rollover = num3 = %d\r",num3
1281	// Printf "num0 + num2 = %d\r",num0+num2
1282
1283	// dispStr will be displayed on the panel
1284	fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
1285
1286	sprintf tmpStr, "%s: %d total bytes\r",fileStr,totBytes
1287	dispStr = tmpStr
1288	sprintf tmpStr,"numXYevents = %d\r",numXYevents
1289	dispStr += tmpStr
1290	sprintf tmpStr,"PP = %d : ",numPP
1291	dispStr += tmpStr
1292	sprintf tmpStr,"ZeroData = %d\r",numZero
1293	dispStr += tmpStr
1294	sprintf tmpStr,"Rollover = %d",num3
1295	dispStr += tmpStr
1296
1297	// /I/U is unsigned 32-bit integer (for the time)
1298	// /B/U is unsigned 8-bit integer (max val=255) for the x and y values
1299
1300	Make/O/U/N=(numXYevents) xLoc,yLoc
1301	Make/O/D/N=(numXYevents) timePt
1302	//// Make/O/U/N=(numXYevents/decFac) xLoc,yLoc
1303	//// Make/O/D/N=(numXYevents/decFac) timePt
1304	// Make/O/U/N=(totBytes/4) xLoc,yLoc //too large, trim when done (bad idea)
1305	// Make/O/D/N=(totBytes/4) timePt
1306	Make/O/D/N=1000 badTimePt,badEventNum,PPTime,PPEventNum,T0Time,T0EventNum
1307	badTimePt=0
1308	badEventNum=0
1309	PPTime=0
1310	PPEventNum=0
1311	T0Time=0
1312	T0EventNum=0
1313	xLoc=0
1314	yLoc=0
1315	timePt=0
1316
1317	nRoll = 0 //number of rollover events
1318	roll_time = 2^26 //units of 10-7 sec
1319
1320	NVAR removeBadEvents = root:Packages:NIST:gRemoveBadEvents
1321
1322	time_msw=0
1323
1324	tic()
1325
1326	ii = 0 //indexes the points in xLoc,yLoc,timePt
1327	//// keep = decFac //keep the first point
1328
1329
1330	Open/R fileref as filepathstr
1331
1332	// remove events at the beginning up to a type==2 so that the msw and lsw times are reset properly
1333	if(RemoveBadEvents == 1)
1334	do
1335	do
1336	FReadLine fileref, buffer //skip the "blank" lines that have one character
1337	while(strlen(buffer) == 1)
1338
1339	if (strlen(buffer) == 0)
1340	break
1341	endif
1342
1343	sscanf buffer,"%x",dataval
1344	// two most sig bits (31-30)
1345	type = (dataval & 0xC0000000)/1073741824 //right shift by 2^30
1346
1347	if(type == 2)
1348	// this is the first event with a proper time value, so process the XY-time event as ususal
1349	// and then break to drop to the main loop, where the next event == type 1
1350
1351	xval = xMax - (dataval & 255) //last 8 bits (7-0)
1352	yval = (dataval & 65280)/256 //bits 15-8, right shift by 2^8
1353
1354	time_lsw = (dataval & 536805376)/65536 //13 bits, 28-16, right shift by 2^16
1355
1356	if(verbose)
1357	// printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1358	printf "%u : %u : %u : %u\r",dataval,timeval,xval,yval
1359	endif
1360
1361	// this is the first point, be sure that ii = 0, and always keep this point
1362	//// if(keep==decFac)
1363	ii = 0
1364	xLoc[ii] = xval
1365	yLoc[ii] = yval
1366	//// keep = 0
1367	//// endif
1368	Print "At beginning of file, numBad = ",numBad
1369	break // the next do loop processes the bulk of the file (** the next event == type 1 = MIR)
1370	else
1371	numBad += 1
1372	numRemoved += 1
1373	endif
1374
1375	//ii+=1 don't increment the counter
1376	while(1)
1377	endif
1378
1379	// now read the main portion of the file.
1380	//// // keep is = 0 if bad points were removed, or is decFac is I need to keep the first point
1381	do
1382	do
1383	FReadLine fileref, buffer //skip the "blank" lines that have one character
1384	while(strlen(buffer) == 1)
1385
1386	if (strlen(buffer) == 0) // this marks the end of the file and is our only way out
1387	break
1388	endif
1389
1390	sscanf buffer,"%x",dataval
1391
1392
1393	// type = (dataval & ~(2^32 - 2^30 -1))/2^30
1394
1395	// two most sig bits (31-30)
1396	type = (dataval & 0xC0000000)/1073741824 //right shift by 2^30
1397
1398	//
1399	// The defintions of the event types
1400	//
1401	//Constant ATXY = 0
1402	//Constant ATXYM = 2
1403	//Constant ATMIR = 1
1404	//Constant ATMAR = 3
1405	//
1406
1407	if(verbose > 0)
1408	verbose -= 1
1409	endif
1410	//
1411	switch(type)
1412	case ATXY: // 0
1413	if(verbose)
1414	printf "XY : "
1415	endif
1416
1417	// if the datavalue is == 0, just skip it now (it can only be interpreted as type 0, obviously)
1418	if(dataval == 0 && RemoveBadEvents == 1)
1419	numRemoved += 1
1420	//Print "zero at ii= ",ii
1421	break //don't increment ii
1422	endif
1423
1424	// if it's a pileup event, skip it now (this can be either type 0 or 2)
1425	bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1426	if(bit29 == 1 && RemoveBadEvents == 1)
1427	PPTime[tmpPP] = timeval
1428	PPEventNum[tmpPP] = ii
1429	tmpPP += 1
1430	numRemoved += 1
1431	break //don't increment ii
1432	endif
1433
1434	// xval = ~(dataval & ~(2^32 - 2^8)) & 127
1435	// yval = ((dataval & ~(2^32 - 2^16 ))/2^8) & 127
1436	// time_lsw = (dataval & ~(2^32 - 2^29))/2^16
1437
1438	xval = xMax - (dataval & 255) //last 8 bits (7-0)
1439	yval = (dataval & 65280)/256 //bits 15-8, right shift by 2^8
1440	time_lsw = (dataval & 536805376)/65536 //13 bits, 28-16, right shift by 2^16
1441
1442	timeval = trunc( nRollroll_time + (time_msw (8192)) + time_lsw ) //left shift msw by 2^13, then add in lsw, as an integer
1443	if (timeval > t_longest)
1444	t_longest = timeval
1445	endif
1446
1447
1448	// catch the "bad" events:
1449	// if an XY event follows a rollover, time_msw is 0 by definition, but does not immediately get
1450	// re-evalulated here. Throw out only the immediately following points where msw is still 8191
1451	if(rolloverHappened && RemoveBadEvents == 1)
1452	// maybe a bad event
1453	if(time_msw == 8191)
1454	badTimePt[numBad] = timeVal
1455	badEventNum[numBad] = ii
1456	numBad +=1
1457	numRemoved += 1
1458	else
1459	// time_msw has been reset, points are good now, so keep this one
1460	//// if(keep==decFac)
1461	xLoc[ii] = xval
1462	yLoc[ii] = yval
1463	timePt[ii] = timeval
1464
1465	// if(xval == 127 && yval == 0)
1466	// // check bit 29
1467	// bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1468	// Print "XY=127,0 : bit29 = ",bit29
1469	// endif
1470
1471	ii+=1
1472	rolloverHappened = 0
1473	//// keep = 0
1474	//// else
1475	//// keep += 1
1476	//// endif
1477	endif
1478	else
1479	// normal processing of good point, keep it
1480	//// if(keep==decFac)
1481	xLoc[ii] = xval
1482	yLoc[ii] = yval
1483	timePt[ii] = timeval
1484
1485	// if(xval == 127 && yval == 0)
1486	// // check bit 29
1487	// bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1488	// Printf "XY=127,0 : bit29 = %u : d=%u\r",bit29,dataval
1489	// endif
1490	ii+=1
1491	//// keep = 0
1492	//// else
1493	//// keep += 1
1494	//// endif
1495	endif
1496
1497
1498	if(verbose)
1499	// printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1500	printf "d=%u : t=%u : msw=%u : lsw=%u : %u : %u \r",dataval,timeval,time_msw,time_lsw,xval,yval
1501	endif
1502
1503	// verbose = 0
1504	break
1505	case ATXYM: // 2
1506	if(verbose)
1507	printf "XYM : "
1508	endif
1509
1510	// if it's a pileup event, skip it now (this can be either type 0 or 2)
1511	// - but can I do this if this is an XY-time event? This will lead to a wrong time, and a time
1512	// assigned to an XY (0,0)...
1513	// bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1514	// if(bit29 == 1 && RemoveBadEvents == 1)
1515	// Print "*****Bit 29 (PP) event set for Type==2, but not handled, ii = ",ii
1516	//// break //don't increment ii
1517	// endif
1518
1519	// xval = ~(dataval & ~(2^32 - 2^8)) & 127
1520	// yval = ((dataval & ~(2^32 - 2^16 ))/2^8) & 127
1521	// time_lsw = (dataval & ~(2^32 - 2^29 ))/2^16 //this method gives a FP result!! likely since the "^" operation gives FP result...
1522
1523	xval = xMax - (dataval & 255) //last 8 bits (7-0)
1524	yval = (dataval & 65280)/256 //bits 15-8, right shift by 2^8
1525
1526	time_lsw = (dataval & 536805376)/65536 //13 bits, 28-16, right shift by 2^16 (result is integer)
1527
1528	if(verbose)
1529	// printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1530	printf "%u : %u : %u : %u\r",dataval,timeval,xval,yval
1531	endif
1532
1533	//// if(keep==decFac) //don't reset keep yet, do this only when ii increments
1534	xLoc[ii] = xval
1535	yLoc[ii] = yval
1536	//// endif
1537
1538	// don't fill in the time yet, or increment the index ii
1539	// the next event MUST be ATMIR with the MSW time bits
1540	//
1541	// verbose = 0
1542	break
1543	case ATMIR: // 1
1544	if(verbose)
1545	printf "MIR : "
1546	endif
1547
1548	time_msw = (dataval & 536805376)/65536 //13 bits, 28-16, right shift by 2^16
1549	timeval = trunc( nRollroll_time + (time_msw (8192)) + time_lsw )
1550	if (timeval > t_longest)
1551	t_longest = timeval
1552	endif
1553	if(verbose)
1554	// printf "%u : %u : %u : %u\r",dataval,time_lsw,time_msw,timeval
1555	printf "d=%u : t=%u : msw=%u : lsw=%u : tlong=%u\r",dataval,timeval,time_msw,time_lsw,t_longest
1556	endif
1557
1558	// the XY position was in the previous event ATXYM
1559	//// if(keep == decFac)
1560	timePt[ii] = timeval
1561	//// endif
1562
1563	bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1564	if(bit29 != 0) // bit 29 set is a T0 event
1565	//Printf "bit29 = 1 at ii = %d : type = %d\r",ii,type
1566	T0Time[tmpT0] = timeval
1567	T0EventNum[tmpT0] = ii
1568	tmpT0 += 1
1569	// reset nRoll = 0 for calcluating the time
1570	nRoll = 0
1571	endif
1572
1573	//// if(keep == decFac)
1574	ii+=1
1575	//// keep = 0
1576	//// endif
1577	// verbose = 0
1578	break
1579	case ATMAR: // 3
1580	if(verbose3)
1581	// verbose = 15
1582	// verbose = 2
1583	printf "MAR : "
1584	endif
1585
1586	// do something with the rollover event?
1587
1588	// check bit 29
1589	bit29 = (dataval & 0x20000000)/536870912 //bit 29 only , shift by 2^29
1590	nRoll += 1
1591	// not doing anything with these bits yet
1592	bit28 = (dataval & 0x10000000)/268435456 //bit 28 only, shift by 2^28
1593	bit27 = (dataval & 0x08000000)/134217728 //bit 27 only, shift by 2^27
1594
1595	if(verbose3)
1596	printf "d=%u : b29=%u : b28=%u : b27=%u : #Roll=%u \r",dataval,bit29, bit28, bit27,nRoll
1597	endif
1598
1599	if(bit29 != 0) // bit 29 set is a T0 event
1600	//Printf "bit29 = 1 at ii = %d : type = %d\r",ii,type
1601	T0Time[tmpT0] = timeval
1602	T0EventNum[tmpT0] = ii
1603	tmpT0 += 1
1604	// reset nRoll = 0 for calcluating the time
1605	nRoll = 0
1606	endif
1607
1608	rolloverHappened = 1
1609
1610	break
1611	endswitch
1612
1613	// if(ii<18)
1614	// printf "TYPE=%d : ii=%d : d=%u : t=%u : msw=%u : lsw=%u : %u : %u \r",type,ii,dataval,timeval,time_msw,time_lsw,xval,yval
1615	// endif
1616
1617	while(1)
1618
1619	Close fileref
1620
1621	printf("Igor full file read done in ")
1622	toc()
1623
1624	Print "Events removed (Igor) = ",numRemoved
1625
1626	sPrintf tmpStr,"\rBad Rollover Events = %d (%4.4g %% of events)",numBad,numBad/numXYevents*100
1627	dispStr += tmpStr
1628	sPrintf tmpStr,"\rTotal Events Removed = %d (%4.4g %% of events)",numRemoved,numRemoved/numXYevents*100
1629	dispStr += tmpStr
1630	SetDataFolder root:
1631
1632	return(0)
1633
1634	End
1635
1636	//////////////
1637	//
1638	// This calls the XOP, as an operation to load the events
1639	//
1640	// -- it's about 35x faster than the Igor code, so I guess that's OK.
1641	//
1642	// conditional compile the whole inner workings in case XOP is not present
1643	Function LoadEvents_XOP()
1644	#if (exists("EventLoadWave")==4)
1645
1646	// NVAR time_msw = root:Packages:NIST:gEvent_time_msw
1647	// NVAR time_lsw = root:Packages:NIST:gEvent_time_lsw
1648	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
1649
1650	SVAR filepathstr = root:Packages:NIST:gEvent_logfile
1651	SVAR dispStr = root:Packages:NIST:gEventDisplayString
1652
1653	SetDataFolder root:Packages:NIST:Event
1654
1655
1656
1657	Variable fileref
1658	String buffer
1659	String fileStr,tmpStr
1660	Variable dataval,timeval,type,numLines,verbose,verbose3
1661	Variable xval,yval,rollBit,nRoll,roll_time,bit29,bit28,bit27
1662	Variable ii,flaggedEvent,rolloverHappened,numBad=0,tmpPP=0,tmpT0=0
1663	Variable Xmax, yMax
1664
1665	xMax = 127 // number the detector from 0->127
1666	yMax = 127
1667
1668	numLines = 0
1669
1670	//Have to declare local variables for Loadwave so that this compiles without XOP.
1671	String S_waveNames
1672	// and those for the XOP
1673	Variable V_nXYevents,V_num1,V_num2,V_num3,V_num0,V_totBytes,V_numPP,V_numT0,V_numDL,V_numFF,V_numZero
1674	Variable V_numBad,V_numRemoved
1675
1676	// what I really need is the number of XY events
1677	Variable numXYevents,num1,num2,num3,num0,totBytes,numPP,numT0,numDL,numFF,numZero
1678	Variable numRemoved
1679	numXYevents = 0
1680	num0 = 0
1681	num1 = 0
1682	num2 = 0
1683	num3 = 0
1684	numPP = 0
1685	numT0 = 0
1686	numDL = 0
1687	numFF = 0
1688	numZero = 0
1689	numRemoved = 0
1690
1691	// get the total number of bytes in the file
1692	Open/R fileref as filepathstr
1693	FStatus fileref
1694	Close fileref
1695
1696	totBytes = V_logEOF
1697	Print "total bytes = ", totBytes
1698
1699	//
1700	// Print "scan only"
1701	// tic()
1702	// EventLoadWave/R/N=EventWave/W filepathstr
1703	// toc()
1704
1705	////
1706	//
1707	// use the XOP operation to load in the data
1708	// -- this does everything - the pre-scan and creating the waves
1709	//
1710	// need to zero the waves before loading, just in case
1711	//
1712
1713	NVAR removeBadEvents = root:Packages:NIST:gRemoveBadEvents
1714
1715	tic()
1716
1717	// Wave/Z wave0=wave0
1718	// Wave/Z wave1=wave1
1719	// Wave/Z wave2=wave2
1720	//
1721	// if(WaveExists(wave0))
1722	// MultiThread wave0=0
1723	// endif
1724	// if(WaveExists(wave1))
1725	// MultiThread wave1=0
1726	// endif
1727	// if(WaveExists(wave2))
1728	// MultiThread wave2=0
1729	// endif
1730
1731	if(removeBadEvents)
1732	EventLoadWave/R/N=EventWave filepathstr
1733	else
1734	EventLoadWave/N=EventWave filepathstr
1735	endif
1736
1737
1738	Print "XOP files loaded = ",S_waveNames
1739
1740	//// -- copy the waves over to xLoc,yLoc,timePt
1741	Wave/Z EventWave0=EventWave0
1742	Wave/Z EventWave1=EventWave1
1743	Wave/Z EventWave2=EventWave2
1744
1745
1746	Duplicate/O EventWave0,xLoc
1747	KillWaves/Z EventWave0
1748
1749	Duplicate/O EventWave1,yLoc
1750	KillWaves/Z EventWave1
1751
1752	Duplicate/O EventWave2,timePt
1753	KillWaves/Z EventWave2
1754
1755	// could do this, but rescaled time will neeed to be converted to SP (or DP)
1756	// and Igor loader was written with Make generating SP/DP waves
1757	// /I/U is unsigned 32-bit integer (for the time)
1758	// /B/U is unsigned 8-bit integer (max val=255) for the x and y values
1759
1760	// Redimension/B/U xLoc,yLoc
1761	// Redimension/I/U timePt
1762
1763	// access the variables from the XOP
1764	numT0 = V_numT0
1765	numPP = V_numPP
1766	num0 = V_num0
1767	num1 = V_num1
1768	num2 = V_num2
1769	num3 = V_num3
1770	numXYevents = V_nXYevents
1771	numZero = V_numZero
1772	numBad = V_numBad
1773	numRemoved = V_numRemoved
1774
1775	Print "(XOP) numT0 = ",numT0
1776	Print "num0 = ",num0
1777	Print "num1 = ",num1
1778	Print "num2 = ",num2
1779	Print "num3 = ",num3
1780
1781
1782	// dispStr will be displayed on the panel
1783	fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
1784
1785	sprintf tmpStr, "%s: %d total bytes\r",fileStr,totBytes
1786	dispStr = tmpStr
1787	sprintf tmpStr,"numXYevents = %d\r",numXYevents
1788	dispStr += tmpStr
1789	sprintf tmpStr,"PP = %d : ",numPP
1790	dispStr += tmpStr
1791	sprintf tmpStr,"ZeroData = %d\r",numZero
1792	dispStr += tmpStr
1793	sprintf tmpStr,"Rollover = %d",num3
1794	dispStr += tmpStr
1795
1796	toc()
1797
1798	Print "Events removed (XOP) = ",numRemoved
1799
1800	sPrintf tmpStr,"\rBad Rollover Events = %d (%4.4g %% of events)",numBad,numBad/numXYevents*100
1801	dispStr += tmpStr
1802	sPrintf tmpStr,"\rTotal Events Removed = %d (%4.4g %% of events)",numRemoved,numRemoved/numXYevents*100
1803	dispStr += tmpStr
1804
1805	SetDataFolder root:
1806
1807	#endif
1808	return(0)
1809
1810	End
1811
1812	//////////////
1813
1814	Proc BinEventBarGraph()
1815
1816	DoWindow/F EventBarGraph
1817	if(V_flag == 0)
1818	PauseUpdate; Silent 1 // building window...
1819	String fldrSav0= GetDataFolder(1)
1820	SetDataFolder root:Packages:NIST:Event:
1821	Display /W=(110,705,610,1132)/N=EventBarGraph /K=1 binCount vs binEndTime
1822	SetDataFolder fldrSav0
1823	ModifyGraph mode=5
1824	ModifyGraph marker=19
1825	ModifyGraph lSize=2
1826	ModifyGraph rgb=(0,0,0)
1827	ModifyGraph msize=2
1828	ModifyGraph hbFill=2
1829	ModifyGraph gaps=0
1830	ModifyGraph usePlusRGB=1
1831	ModifyGraph toMode=1
1832	ModifyGraph useBarStrokeRGB=1
1833	ModifyGraph standoff=0
1834	SetAxis left 0,*
1835	Label bottom "\\Z14Time (seconds)"
1836	Label left "\\Z14Number of Events"
1837	endif
1838	End
1839
1840
1841	Proc ShowBinTable()
1842
1843	DoWindow/F BinEventTable
1844	if(V_flag == 0)
1845	PauseUpdate; Silent 1 // building window...
1846	String fldrSav0= GetDataFolder(1)
1847	SetDataFolder root:Packages:NIST:Event:
1848	Edit/W=(498,699,1003,955) /K=1/N=BinEventTable binCount,binEndTime,timeWidth
1849	ModifyTable format(Point)=1,sigDigits(binEndTime)=8,width(binEndTime)=100
1850	SetDataFolder fldrSav0
1851	endif
1852	EndMacro
1853
1854
1855	// only show the first 1500 data points
1856	//
1857	Proc ShowRescaledTimeGraph()
1858
1859	DoWindow/F RescaledTimeGraph
1860	if(V_flag == 0)
1861	PauseUpdate; Silent 1 // building window...
1862	String fldrSav0= GetDataFolder(1)
1863	SetDataFolder root:Packages:NIST:Event:
1864	Display /W=(25,44,486,356)/K=1/N=RescaledTimeGraph rescaledTime
1865	SetDataFolder fldrSav0
1866	ModifyGraph mode=4
1867	ModifyGraph marker=19
1868	ModifyGraph rgb(rescaledTime)=(0,0,0)
1869	ModifyGraph msize=1
1870	// SetAxis/A=2 left //only autoscale the visible data (based on the bottom limits)
1871	SetAxis bottom 0,1500
1872	ErrorBars rescaledTime OFF
1873	Label left "\\Z14Time (seconds)"
1874	Label bottom "\\Z14Event number"
1875	ShowInfo
1876	endif
1877
1878	EndMacro
1879
1880
1881
1882	Proc ExportSlicesAsVAX(firstNum,prefix)
1883	Variable firstNum=1
1884	String prefix="SAMPL"
1885
1886	SaveSlicesAsVAX(firstNum,prefix[0,4]) //make sure that the prefix is 5 chars
1887	End
1888
1889	//////// procedures to be able to export the slices as RAW VAX files.
1890	//
1891	// 1- load the raw data file to use the header (it must already be in RAW)
1892	// 1.5- copy the raw data to the temp folder (STO)
1893	// 1.7- ask for the prefix and starting run number (these are passed in)
1894	// 2- copy the slice of data to the temp folder (STO)
1895	// 3- touch up the time/counts in the slice header values in STO
1896	// 4- write out the VAX file
1897	// 5- repeat (2-4) for the number of slices
1898	//
1899	//
1900	Function SaveSlicesAsVAX(firstNum,prefix)
1901	Variable firstNum
1902	String prefix
1903
1904	DoAlert 1,"Is the full data file loaded as a RAW data file? If not, load it and start over..."
1905	if(V_flag == 2)
1906	return (0)
1907	endif
1908
1909	// copy the contents of RAW to STO so I can work from there
1910	CopyWorkContents("RAW","STO")
1911
1912	// now declare all of the waves, now that they are sure to be there
1913
1914	WAVE slicedData=root:Packages:NIST:Event:slicedData
1915	Make/O/D/N=(128,128) curSlice
1916
1917	NVAR nslices = root:Packages:NIST:gEvent_nslices
1918	WAVE binEndTime = root:Packages:NIST:Event:binEndTime
1919
1920	Wave rw=root:Packages:NIST:STO:realsRead
1921	Wave iw=root:Packages:NIST:STO:integersRead
1922	Wave/T tw=root:Packages:NIST:STO:textRead
1923	Wave data=root:Packages:NIST:STO:data
1924	Wave linear_data=root:Packages:NIST:STO:linear_data
1925
1926
1927	Wave rw_raw=root:Packages:NIST:RAW:realsRead
1928	Wave iw_raw=root:Packages:NIST:RAW:integersRead
1929	Wave/T tw_raw=root:Packages:NIST:RAW:textRead
1930
1931	// for generating the alphanumeric
1932	String timeStr= secs2date(datetime,-1)
1933	String monthStr=StringFromList(1, timeStr ,"/")
1934	String numStr="",labelStr
1935
1936	Variable ii,err,binFraction
1937
1938	for(ii=0;ii<nslices;ii+=1)
1939
1940	//get the current slice and put it in the STO folder
1941	curSlice = slicedData[p][q][ii]
1942	data = curSlice
1943	linear_data = curSlice
1944
1945	// touch up the header as needed
1946	// count time = iw[2]
1947	// monCt = rw[0]
1948	// detCt = rw[2]
1949	//tw[0] must now be the file name
1950	//
1951	// count time = fraction of total binning * total count time
1952	binFraction = (binEndTime[ii+1]-binEndTime[ii])/(binEndTime[nslices]-binEndTime[0])
1953
1954	iw[2] = trunc(binFraction*iw_raw[2])
1955	rw[0] = trunc(binFraction*rw_raw[0])
1956	rw[2] = sum(curSlice,-inf,inf) //total counts in slice
1957
1958	if(firstNum<10)
1959	numStr = "00"+num2str(firstNum)
1960	else
1961	if(firstNum<100)
1962	numStr = "0"+num2str(firstNum)
1963	else
1964	numStr = num2str(firstNum)
1965	Endif
1966	Endif
1967	tw[0] = prefix+numstr+".SA2_EVE_"+(num2char(str2num(monthStr)+64))+numStr
1968	labelStr = tw_raw[6]
1969
1970	labelStr = PadString(labelStr,60,0x20) //60 fortran-style spaces
1971	tw[6] = labelStr[0,59]
1972
1973	//write out the file - this uses the tw[0] and home path
1974	Write_VAXRaw_Data("STO","",0)
1975
1976	//increment the run number, alpha
1977	firstNum += 1
1978	endfor
1979
1980	return(0)
1981	End
1982
1983
1984
1985
1986
1987	/////////////
1988	//The histogramming
1989	//
1990	// 6 AUG 2012
1991	//
1992	// from Igor Exchange, RGerkin
1993	// http://www.igorexchange.com/node/1373
1994	// -- see the related thread on the mailing list
1995	//
1996	//Function Setup_JointHistogram()
1997	//
1998	//// tic()
1999	//
2000	// make/D /o/n=1000000 data1=gnoise(1), data2=gnoise(1)
2001	// make/D /o/n=(25,25) myHist
2002	// setscale x,-3,3,myHist
2003	// setscale y,-3,3,myHist
2004	// IndexForHistogram(data1,data2,myhist)
2005	// Wave index=SavedIndex
2006	// JointHistogram(data1,data2,myHist,index)
2007	// NewImage myHist
2008	//
2009	//// toc()
2010	//
2011	//End
2012
2013
2014	Function JointHistogram(w0,w1,hist,index)
2015	wave w0,w1,hist,index
2016
2017	variable bins0=dimsize(hist,0)
2018	variable bins1=dimsize(hist,1)
2019	variable n=numpnts(w0)
2020	variable left0=dimoffset(hist,0)
2021	variable left1=dimoffset(hist,1)
2022	variable right0=left0+bins0*dimdelta(hist,0)
2023	variable right1=left1+bins1*dimdelta(hist,1)
2024
2025	// Compute the histogram and redimension it.
2026	histogram /b={0,1,bins0*bins1} index,hist
2027	redimension/D /n=(bins0,bins1) hist // Redimension to 2D.
2028	setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension.
2029	setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension.
2030	End
2031
2032
2033	// histogram with a point range
2034	//
2035	// x- just need to send x2pnt or findLevel, or something similar to define the POINT
2036	// values
2037	//
2038	// x- can also speed this up since the index only needs to be done once, so the
2039	// histogram operation can be done separately, as the bins require
2040	//
2041	//
2042	Function JointHistogramWithRange(w0,w1,hist,index,pt1,pt2)
2043	wave w0,w1,hist,index
2044	Variable pt1,pt2
2045
2046	variable bins0=dimsize(hist,0)
2047	variable bins1=dimsize(hist,1)
2048	variable n=numpnts(w0)
2049	variable left0=dimoffset(hist,0)
2050	variable left1=dimoffset(hist,1)
2051	variable right0=left0+bins0*dimdelta(hist,0)
2052	variable right1=left1+bins1*dimdelta(hist,1)
2053
2054	// Compute the histogram and redimension it.
2055	histogram /b={0,1,bins0*bins1}/R=[pt1,pt2] index,hist
2056	redimension/D /n=(bins0,bins1) hist // Redimension to 2D.
2057	setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension.
2058	setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension.
2059	End
2060
2061
2062	// just does the indexing, creates wave SavedIndex in the current folder for the index
2063	//
2064	Function IndexForHistogram(w0,w1,hist)
2065	wave w0,w1,hist
2066
2067	variable bins0=dimsize(hist,0)
2068	variable bins1=dimsize(hist,1)
2069	variable n=numpnts(w0)
2070	variable left0=dimoffset(hist,0)
2071	variable left1=dimoffset(hist,1)
2072	variable right0=left0+bins0*dimdelta(hist,0)
2073	variable right1=left1+bins1*dimdelta(hist,1)
2074
2075	// Scale between 0 and the number of bins to create an index wave.
2076	if(ThreadProcessorCount<4) // For older machines, matrixop is faster.
2077	matrixop /free idx=round(bins0(w0-left0)/(right0-left0))+bins0round(bins1*(w1-left1)/(right1-left1))
2078	else // For newer machines with many cores, multithreading with make is faster.
2079	make/free/n=(n) idx
2080	multithread idx=round(bins0(w0-left0)/(right0-left0))+bins0round(bins1*(w1-left1)/(right1-left1))
2081	endif
2082
2083	KillWaves/Z SavedIndex
2084	MoveWave idx,SavedIndex
2085
2086	// // Compute the histogram and redimension it.
2087	// histogram /b={0,1,bins0*bins1} idx,hist
2088	// redimension /n=(bins0,bins1) hist // Redimension to 2D.
2089	// setscale x,left0,right0,hist // Fix the histogram scaling in the x-dimension.
2090	// setscale y,left1,right1,hist // Fix the histogram scaling in the y-dimension.
2091	End
2092
2093
2094
2095
2096
2097	////////////// Post-processing of the event mode data
2098	Proc ShowEventCorrectionPanel()
2099	DoWindow/F EventCorrectionPanel
2100	if(V_flag ==0)
2101	EventCorrectionPanel()
2102	EndIf
2103	End
2104
2105	Proc EventCorrectionPanel()
2106
2107	PauseUpdate; Silent 1 // building window...
2108	SetDataFolder root:Packages:NIST:Event:
2109
2110	Display /W=(35,44,761,533)/K=2 rescaledTime
2111	DoWindow/C EventCorrectionPanel
2112	ModifyGraph mode=4
2113	ModifyGraph marker=19
2114	ModifyGraph rgb=(0,0,0)
2115	ModifyGraph msize=1
2116	ErrorBars rescaledTime OFF
2117	Label left "\\Z14Time (seconds)"
2118	Label bottom "\\Z14Event number"
2119	SetAxis bottom 0,0.10*numpnts(rescaledTime) //show 1st 10% of data for speed in displaying
2120
2121	ControlBar 100
2122	Button button0,pos={18,12},size={70,20},proc=EC_AddCursorButtonProc,title="Cursors"
2123	Button button1,pos={153,12},size={80,20},proc=EC_AddTimeButtonProc,title="Add time"
2124	Button button2,pos={153,38},size={80,20},proc=EC_SubtractTimeButtonProc,title="Subtr time"
2125	Button button3,pos={153,64},size={90,20},proc=EC_TrimPointsButtonProc,title="Trim points"
2126	Button button4,pos={295+200,12},size={90,20},proc=EC_SaveWavesButtonProc,title="Save Waves"
2127	Button button5,pos={295+200,38},size={100,20},proc=EC_ImportWavesButtonProc,title="Import Waves"
2128	Button button6,pos={18,38},size={80,20},proc=EC_ShowAllButtonProc,title="All Data"
2129	Button button7,pos={683,12},size={30,20},proc=EC_HelpButtonProc,title="?"
2130	Button button8,pos={658,72},size={60,20},proc=EC_DoneButtonProc,title="Done"
2131
2132	Button button9,pos={295,12},size={110,20},proc=EC_FindStepButton_down,title="Find Step Down"
2133	Button button10,pos={295,38},size={110,20},proc=EC_FindStepButton_up,title="Find Step Up"
2134	Button button11,pos={295,64},size={110,20},proc=EC_DoDifferential,title="Differential"
2135
2136	SetDataFolder root:
2137
2138	EndMacro
2139
2140	Function EC_AddCursorButtonProc(ba) : ButtonControl
2141	STRUCT WMButtonAction &ba
2142
2143	switch( ba.eventCode )
2144	case 2: // mouse up
2145	// click code here
2146	SetDataFolder root:Packages:NIST:Event:
2147
2148	Wave rescaledTime = rescaledTime
2149	Cursor/P A rescaledTime 0
2150	Cursor/P B rescaledTime numpnts(rescaledTime)-1
2151	ShowInfo
2152	SetDataFolder root:
2153	break
2154	case -1: // control being killed
2155	break
2156	endswitch
2157
2158	return 0
2159	End
2160
2161	Function EC_AddTimeButtonProc(ba) : ButtonControl
2162	STRUCT WMButtonAction &ba
2163
2164	switch( ba.eventCode )
2165	case 2: // mouse up
2166	// click code here
2167	SetDataFolder root:Packages:NIST:Event:
2168
2169	Wave rescaledTime = rescaledTime
2170	Wave timePt = timePt
2171	Variable rollTime,rollTicks,ptA,ptB,lo,hi
2172
2173	rollTicks = 2^26 // in ticks
2174	rollTime = 2^26*1e-7 // in seconds
2175	ptA = pcsr(A)
2176	ptB = pcsr(B)
2177	lo=min(ptA,ptB)
2178	hi=max(ptA,ptB)
2179
2180	MultiThread timePt[lo,hi] += rollTicks
2181	MultiThread rescaledTime[lo,hi] += rollTime
2182
2183
2184	SetDataFolder root:
2185	break
2186	case -1: // control being killed
2187	break
2188	endswitch
2189
2190	return 0
2191	End
2192
2193	Function EC_SubtractTimeButtonProc(ba) : ButtonControl
2194	STRUCT WMButtonAction &ba
2195
2196	switch( ba.eventCode )
2197	case 2: // mouse up
2198	// click code here
2199	SetDataFolder root:Packages:NIST:Event:
2200
2201	Wave rescaledTime = rescaledTime
2202	Wave timePt = timePt
2203	Variable rollTime,rollTicks,ptA,ptB,lo,hi
2204
2205	rollTicks = 2^26 // in ticks
2206	rollTime = 2^26*1e-7 // in seconds
2207	ptA = pcsr(A)
2208	ptB = pcsr(B)
2209	lo=min(ptA,ptB)
2210	hi=max(ptA,ptB)
2211
2212	MultiThread timePt[lo,hi] -= rollTicks
2213	MultiThread rescaledTime[lo,hi] -= rollTime
2214
2215	SetDataFolder root:
2216
2217	break
2218	case -1: // control being killed
2219	break
2220	endswitch
2221
2222	return 0
2223	End
2224
2225	// points removed are inclusive
2226	//
2227	// put both cursors on the same point to remove just that single point
2228	//
2229	Function EC_TrimPointsButtonProc(ba) : ButtonControl
2230	STRUCT WMButtonAction &ba
2231
2232	switch( ba.eventCode )
2233	case 2: // mouse up
2234	// click code here
2235	SetDataFolder root:Packages:NIST:Event:
2236
2237	Wave rescaledTime = rescaledTime
2238	Wave timePt = timePt
2239	Wave xLoc = xLoc
2240	Wave yLoc = yLoc
2241	Variable rollTime,ptA,ptB,numElements,lo,hi
2242
2243	rollTime = 2^26*1e-7 // in seconds
2244	ptA = pcsr(A)
2245	ptB = pcsr(B)
2246	lo=min(ptA,ptB)
2247	hi=max(ptA,ptB)
2248	numElements = abs(ptA-ptB)+1 //so points removed are inclusive
2249	DeletePoints lo, numElements, rescaledTime,timePt,xLoc,yLoc
2250
2251	printf "Points %g to %g have been deleted in rescaledTime, timePt, xLoc, and yLoc\r",ptA,ptB
2252	SetDataFolder root:
2253
2254	break
2255	case -1: // control being killed
2256	break
2257	endswitch
2258
2259	return 0
2260	End
2261
2262	// un-sort the data first, then save it
2263	Function EC_SaveWavesButtonProc(ba) : ButtonControl
2264	STRUCT WMButtonAction &ba
2265
2266	switch( ba.eventCode )
2267	case 2: // mouse up
2268	// click code here
2269
2270	// Execute "UndoTheSorting()"
2271
2272	SetDataFolder root:Packages:NIST:Event:
2273
2274	Wave rescaledTime = rescaledTime
2275	Wave timePt = timePt
2276	Wave xLoc = xLoc
2277	Wave yLoc = yLoc
2278	Save/T xLoc,yLoc,timePt ,rescaledTime //will ask for a name
2279
2280	SetDataFolder root:
2281	break
2282	case -1: // control being killed
2283	break
2284	endswitch
2285
2286	return 0
2287	End
2288
2289	// this duplicates all of the bits that would be done if the "load" button was pressed
2290	//
2291	//
2292	Function EC_ImportWavesButtonProc(ba) : ButtonControl
2293	STRUCT WMButtonAction &ba
2294
2295	switch( ba.eventCode )
2296	case 2: // mouse up
2297	// click code here
2298	SetDataFolder root:Packages:NIST:Event:
2299
2300	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
2301	SVAR dispStr = root:Packages:NIST:gEventDisplayString
2302	String tmpStr="",fileStr,filePathStr
2303
2304	// load in the waves, saved as Igor text to preserve the data type
2305	LoadWave/T/O
2306	filePathStr = S_fileName
2307	if(strlen(S_fileName) == 0)
2308	//user cancelled
2309	DoAlert 0,"No file selected, nothing done."
2310	return(0)
2311	endif
2312
2313	NVAR mode = root:Packages:NIST:gEvent_Mode // ==0 for "stream", ==1 for Oscillatory
2314	// clear out the old sort index, if present, since new data is being loaded
2315	KillWaves/Z OscSortIndex
2316	Wave timePt=timePt
2317	Wave rescaledTime=rescaledTime
2318
2319	// Duplicate/O timePt rescaledTime
2320	// if(mode==MODE_STREAM)
2321	// rescaledTime = 1e-7*(timePt-timePt[0]) //convert to seconds and start from zero
2322	// else
2323	// rescaledTime = timePt*1e-7 //just take the times as-is
2324	// endif
2325
2326	t_longest = waveMax(rescaledTime) //should be the last point
2327
2328
2329	fileStr = ParseFilePath(0, filepathstr, ":", 1, 0)
2330	sprintf tmpStr, "%s: a user-modified event file\r",fileStr
2331	dispStr = tmpStr
2332
2333	break
2334	case -1: // control being killed
2335	break
2336	endswitch
2337
2338	return 0
2339	End
2340
2341
2342	Function EC_ShowAllButtonProc(ba) : ButtonControl
2343	STRUCT WMButtonAction &ba
2344
2345	switch( ba.eventCode )
2346	case 2: // mouse up
2347	// click code here
2348	SetAxis/A
2349	break
2350	case -1: // control being killed
2351	break
2352	endswitch
2353
2354	return 0
2355	End
2356
2357	Function EC_HelpButtonProc(ba) : ButtonControl
2358	STRUCT WMButtonAction &ba
2359
2360	switch( ba.eventCode )
2361	case 2: // mouse up
2362	// click code here
2363	DisplayHelpTopic/Z "Event Mode Data[Correcting for things that go wrong]"
2364	break
2365	case -1: // control being killed
2366	break
2367	endswitch
2368
2369	return 0
2370	End
2371
2372	Function EC_DoneButtonProc(ba) : ButtonControl
2373	STRUCT WMButtonAction &ba
2374
2375	switch( ba.eventCode )
2376	case 2: // mouse up
2377	// click code here
2378	DoWindow/K EventCorrectionPanel
2379	break
2380	case -1: // control being killed
2381	break
2382	endswitch
2383
2384	return 0
2385	End
2386
2387	//upDown 5 or -5 looks for spikes +5 or -5 std deviations from mean
2388	Function PutCursorsAtStep(upDown)
2389	Variable upDown
2390
2391	SetDataFolder root:Packages:NIST:Event:
2392
2393	Wave rescaledTime=rescaledTime
2394	Wave rescaledTime_DIF=rescaledTime_DIF
2395	Variable avg,pt,zoom
2396
2397	zoom = 200 //points in each direction
2398
2399	WaveStats/Q rescaledTime_DIF
2400	avg = V_avg
2401
2402
2403	FindLevel/P/Q rescaledTime_DIF avg*upDown
2404	if(V_flag==0)
2405	pt = V_levelX
2406	WaveStats/Q/R=[pt-zoom,pt+zoom] rescaledTime // find the max/min y-vallues within the point range
2407	else
2408	Print "Level not found"
2409	return(0)
2410	endif
2411
2412	Variable loLeft,hiLeft, loBottom,hiBottom
2413	loLeft = V_min*0.98 //+/- 2%
2414	hiLeft = V_max*1.02
2415
2416	SetAxis left loLeft,hiLeft
2417	SetAxis bottom pnt2x(rescaledTime,pt-zoom),pnt2x(rescaledTime,pt+zoom)
2418
2419	Cursor/P A rescaledTime pt+2 //at the point
2420	Cursor/P B rescaledTime numpnts(rescaledTime)-1 //at the end
2421
2422	SetDataFolder root:
2423
2424	return(0)
2425	End
2426
2427
2428	Function EC_FindStepButton_down(ctrlName) : ButtonControl
2429	String ctrlName
2430
2431	Variable upDown = -5
2432	PutCursorsAtStep(upDown)
2433
2434	return(0)
2435	end
2436
2437
2438	Function EC_FindStepButton_up(ctrlName) : ButtonControl
2439	String ctrlName
2440
2441	Variable upDown = 5
2442	PutCursorsAtStep(upDown)
2443
2444	return(0)
2445	end
2446
2447
2448	Function EC_DoDifferential(ctrlName) : ButtonControl
2449	String ctrlName
2450
2451	DifferentiatedTime()
2452	DoWindow/F EventCorrectionPanel
2453
2454	//if trace is not on graph, add it
2455	SetDataFolder root:Packages:NIST:Event:
2456
2457	String list = WaveList("*_DIF", ";", "WIN:EventCorrectionPanel")
2458	if(strlen(list) == 0)
2459	AppendToGraph/R rescaledTime_DIF
2460	ModifyGraph msize=1,rgb(rescaledTime_DIF)=(65535,0,0)
2461	ReorderTraces rescaledTime,{rescaledTime_DIF} // put the differential behind the event data
2462	endif
2463	SetDataFolder root:
2464	return(0)
2465	end
2466
2467	////////////// Custom Bins /////////////////////
2468	//
2469	//
2470	//
2471	// make sure that the bins are defined and the waves exist before
2472	// trying to draw the panel
2473	//
2474	Proc Show_CustomBinPanel()
2475	DoWindow/F CustomBinPanel
2476	if(V_flag ==0)
2477	Init_CustomBins()
2478	CustomBinPanel()
2479	EndIf
2480	End
2481
2482
2483	Function Init_CustomBins()
2484
2485	NVAR nSlice = root:Packages:NIST:gEvent_nslices
2486	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
2487
2488	Variable/G root:Packages:NIST:gEvent_ForceTmaxBin=1 //==1 to enforce t_longest in user-defined custom bins
2489
2490	SetDataFolder root:Packages:NIST:Event:
2491
2492	Make/O/D/N=(nSlice) timeWidth
2493	Make/O/D/N=(nSlice+1) binEndTime,binCount
2494
2495	timeWidth = t_longest/nslice
2496	binEndTime = p
2497	binCount = p+1
2498
2499	SetDataFolder root:
2500
2501	return(0)
2502	End
2503
2504	////////////////
2505	//
2506	// Allow custom definitions of the bin widths
2507	//
2508	// Define by the number of bins, and the time width of each bin
2509	//
2510	// This shares the number of slices and the maximum time with the main panel
2511	//
2512	Proc CustomBinPanel()
2513	PauseUpdate; Silent 1 // building window...
2514	NewPanel /W=(130,44,851,455)/K=2 /N=CustomBinPanel
2515	DoWindow/C CustomBinPanel
2516	ModifyPanel fixedSize=1//,noEdit =1
2517	SetDrawLayer UserBack
2518
2519	Button button0,pos={654,42}, size={50,20},title="Done",fSize=12
2520	Button button0,proc=CB_Done_Proc
2521	Button button1,pos={663,14},size={40,20},proc=CB_HelpButtonProc,title="?"
2522	Button button2,pos={216,42},size={80,20},title="Update",proc=CB_UpdateWavesButton
2523	SetVariable setvar1,pos={23,13},size={160,20},title="Number of slices",fSize=12
2524	SetVariable setvar1,proc=CB_NumSlicesSetVarProc,value=root:Packages:NIST:gEvent_nslices
2525	SetVariable setvar2,pos={24,44},size={160,20},title="Max Time (s)",fSize=10
2526	SetVariable setvar2,value=root:Packages:NIST:gEvent_t_longest
2527
2528	CheckBox chkbox1,pos={216,14},title="Enforce Max Time?"
2529	CheckBox chkbox1,variable = root:Packages:NIST:gEvent_ForceTmaxBin
2530	Button button3,pos={500,14},size={90,20},proc=CB_SaveBinsButtonProc,title="Save Bins"
2531	Button button4,pos={500,42},size={100,20},proc=CB_ImportBinsButtonProc,title="Import Bins"
2532
2533	SetDataFolder root:Packages:NIST:Event:
2534
2535	Display/W=(291,86,706,395)/HOST=CustomBinPanel/N=BarGraph binCount vs binEndTime
2536	ModifyGraph mode=5
2537	ModifyGraph marker=19
2538	ModifyGraph lSize=2
2539	ModifyGraph rgb=(0,0,0)
2540	ModifyGraph msize=2
2541	ModifyGraph hbFill=2
2542	ModifyGraph gaps=0
2543	ModifyGraph usePlusRGB=1
2544	ModifyGraph toMode=1
2545	ModifyGraph useBarStrokeRGB=1
2546	ModifyGraph standoff=0
2547	SetAxis left 0,*
2548	Label bottom "\\Z14Time (seconds)"
2549	Label left "\\Z14Number of Events"
2550	SetActiveSubwindow ##
2551
2552	// and the table
2553	Edit/W=(13,87,280,394)/HOST=CustomBinPanel/N=T0
2554	AppendToTable/W=CustomBinPanel#T0 timeWidth,binEndTime
2555	ModifyTable width(Point)=40
2556	SetActiveSubwindow ##
2557
2558	SetDataFolder root:
2559
2560	EndMacro
2561
2562	// save the bins - use Igor Text format
2563	//
2564	Function CB_SaveBinsButtonProc(ba) : ButtonControl
2565	STRUCT WMButtonAction &ba
2566
2567	switch( ba.eventCode )
2568	case 2: // mouse up
2569	// click code here
2570
2571	SetDataFolder root:Packages:NIST:Event:
2572
2573	Wave timeWidth = timeWidth
2574	Wave binEndTime = binEndTime
2575
2576	Save/T timeWidth,binEndTime //will ask for a name
2577
2578	break
2579	case -1: // control being killed
2580	break
2581	endswitch
2582
2583	SetDataFolder root:
2584
2585	return 0
2586	End
2587
2588	// Import the bins - use Igor Text format
2589	//
2590	// -- be sure that the number of bins is reset
2591	// -?- how about the t_longest? - this should be set by the load, not here
2592	//
2593	// -- loads in timeWidth and binEndTime
2594	//
2595	Function CB_ImportBinsButtonProc(ba) : ButtonControl
2596	STRUCT WMButtonAction &ba
2597
2598	switch( ba.eventCode )
2599	case 2: // mouse up
2600	// click code here
2601	NVAR nSlice = root:Packages:NIST:gEvent_nslices
2602
2603	SetDataFolder root:Packages:NIST:Event:
2604
2605	// prompt for the load of data
2606	LoadWave/T/O
2607	if(strlen(S_fileName) == 0)
2608	//user cancelled
2609	DoAlert 0,"No file selected, nothing done."
2610	return(0)
2611	endif
2612
2613	Wave timeWidth = timeWidth
2614	nSlice = numpnts(timeWidth)
2615
2616	break
2617	case -1: // control being killed
2618	break
2619	endswitch
2620
2621	SetDataFolder root:
2622
2623	return 0
2624	End
2625
2626
2627
2628	//
2629	// can either use the widths as stated -- then the end time may not
2630	// match the actual end time of the data set
2631	//
2632	// -- or --
2633	//
2634	// enforce the end time of the data set to be the end time of the bins,
2635	// then the last bin width must be reset to force the constraint
2636	//
2637	//
2638	Function CB_UpdateWavesButton(ba) : ButtonControl
2639	STRUCT WMButtonAction &ba
2640
2641	switch( ba.eventCode )
2642	case 2: // mouse up
2643	// click code here
2644	NVAR nSlice = root:Packages:NIST:gEvent_nslices
2645	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
2646	NVAR enforceTmax = root:Packages:NIST:gEvent_ForceTmaxBin
2647
2648	// update the waves, and recalculate everything for the display
2649	SetDataFolder root:Packages:NIST:Event:
2650
2651	Wave timeWidth = timeWidth
2652	Wave binEndTime = binEndTime
2653	Wave binCount = binCount
2654
2655	// use the widths as entered
2656	binEndTime[0] = 0
2657	binEndTime[1,] = binEndTime[p-1] + timeWidth[p-1]
2658
2659	// enforce the longest time as the end bin time
2660	// note that this changes the last time width
2661	if(enforceTmax)
2662	binEndTime[nSlice] = t_longest
2663	timeWidth[nSlice-1] = t_longest - binEndTime[nSlice-1]
2664	endif
2665
2666	binCount = p+1
2667	binCount[nSlice] = 0 // last point is zero, just for display
2668	// binCount *= sign(timeWidth) //to alert to negative time bins
2669
2670	// make the timeWidth bold and red if the widths are negative
2671	WaveStats/Q timeWidth
2672	if(V_min < 0)
2673	ModifyTable/W=CustomBinPanel#T0 style(timeWidth)=1,rgb(timeWidth)=(65535,0,0)
2674	else
2675	ModifyTable/W=CustomBinPanel#T0 style(timeWidth)=0,rgb(timeWidth)=(0,0,0)
2676	endif
2677
2678	break
2679	case -1: // control being killed
2680	break
2681	endswitch
2682
2683	SetDataFolder root:
2684
2685	return 0
2686	End
2687
2688	Function CB_HelpButtonProc(ba) : ButtonControl
2689	STRUCT WMButtonAction &ba
2690
2691	switch( ba.eventCode )
2692	case 2: // mouse up
2693	// click code here
2694	DisplayHelpTopic/Z "Event Mode Data[Setting up Custom Bin Widths]"
2695	break
2696	case -1: // control being killed
2697	break
2698	endswitch
2699
2700	return 0
2701	End
2702
2703	Function CB_Done_Proc(ba) : ButtonControl
2704	STRUCT WMButtonAction &ba
2705
2706	String win = ba.win
2707	switch (ba.eventCode)
2708	case 2:
2709	DoWindow/K CustomBinPanel
2710	break
2711	endswitch
2712	return(0)
2713	End
2714
2715
2716	Function CB_NumSlicesSetVarProc(sva) : SetVariableControl
2717	STRUCT WMSetVariableAction &sva
2718
2719	switch( sva.eventCode )
2720	case 1: // mouse up
2721	case 2: // Enter key
2722	case 3: // Live update
2723	Variable dval = sva.dval
2724	String sval = sva.sval
2725	SetDataFolder root:Packages:NIST:Event:
2726
2727	Wave timeWidth = timeWidth
2728	Wave binEndTime = binEndTime
2729
2730	Redimension/N=(dval) timeWidth
2731	Redimension/N=(dval+1) binEndTime,binCount
2732
2733	SetDataFolder root:
2734
2735	break
2736	case -1: // control being killed
2737	break
2738	endswitch
2739
2740	return 0
2741	End
2742
2743
2744	///////////////////
2745	//
2746	// utility to split a large file
2747	// 100 MB is the recommended size
2748	// events can be clipped here, so be sure to trim the ends of the
2749	// resulting files as needed.
2750	//
2751	// - works like the unix 'split' command
2752	//
2753	//
2754
2755	Proc SplitBigFile(splitSize, baseStr)
2756	Variable splitSize = 100
2757	String baseStr="split"
2758	Prompt splitSize,"Target file size, in MB"
2759	Prompt baseStr,"File prefix, number will be appended"
2760
2761
2762	fSplitBigFile(splitSize, baseStr)
2763
2764	ShowSplitFileTable()
2765	End
2766
2767	Function/S fSplitBigFile(splitSize, baseStr)
2768	Variable splitSize
2769	String baseStr
2770
2771
2772	String fileName="" // File name, partial path, full path or "" for dialog.
2773	String pathName=""
2774	Variable refNum
2775	String str
2776	SVAR listStr = root:Packages:NIST:Event:gSplitFileList
2777
2778	listStr="" //initialize output list
2779
2780	Variable readSize=1e6 //1 MB
2781	Make/O/B/U/N=(readSize) aBlob //1MB worth
2782	Variable numSplit
2783	Variable num,ii,jj,outRef,frac
2784	String thePath, outStr
2785
2786	Printf "SplitSize = %u MB\r",splitSize
2787	splitSize = trunc(splitSize) * 1e6 // now in bytes
2788
2789
2790	// Open file for read.
2791	Open/R/Z=2/F="????"/P=$pathName refNum as fileName
2792	thePath = ParseFilePath(1, S_fileName, ":", 1, 0)
2793	Print "thePath = ",thePath
2794
2795	// Store results from Open in a safe place.
2796	Variable err = V_flag
2797	String fullPath = S_fileName
2798
2799	if (err == -1)
2800	Print "cancelled by user."
2801	return ("")
2802	endif
2803
2804	FStatus refNum
2805
2806	Printf "total # bytes = %u\r",V_logEOF
2807
2808	numSplit=0
2809	if(V_logEOF > splitSize)
2810	numSplit = trunc(V_logEOF/splitSize)
2811	endif
2812
2813	frac = V_logEOF - numSplit*splitSize
2814	Print "numSplit = ",numSplit
2815	Printf "frac = %u\r",frac
2816
2817	num=0
2818	if(frac > readSize)
2819	num = trunc(frac/readSize)
2820	endif
2821
2822
2823	frac = frac - num*readSize
2824
2825	Print "num = ",num
2826	Printf "frac = %u\r",frac
2827
2828	// baseStr = "split"
2829
2830	for(ii=0;ii<numSplit;ii+=1)
2831	outStr = (thePath+baseStr+num2str(ii))
2832	// Print "outStr = ",outStr
2833	Open outRef as outStr
2834
2835	for(jj=0;jj<(splitSize/readSize);jj+=1)
2836	FBinRead refNum,aBlob
2837	FBinWrite outRef,aBlob
2838	endfor
2839
2840	Close outRef
2841	listStr += outStr+";"
2842	endfor
2843
2844	Make/O/B/U/N=(frac) leftover
2845	// ii was already incremented past the loop
2846	outStr = (thePath+baseStr+num2str(ii))
2847	Open outRef as outStr
2848	for(jj=0;jj<num;jj+=1)
2849	FBinRead refNum,aBlob
2850	FBinWrite outRef,aBlob
2851	endfor
2852	FBinRead refNum,leftover
2853	FBinWrite outRef,leftover
2854
2855	Close outRef
2856	listStr += outStr+";"
2857
2858
2859	FSetPos refNum,V_logEOF
2860	Close refNum
2861
2862	KillWaves/Z aBlob,leftover
2863	return(listStr)
2864	End
2865
2866	// allows the list of loaded files to be edited
2867	Function ShowSplitFileTable()
2868
2869	SVAR str = root:Packages:NIST:Event:gSplitFileList
2870
2871	Make/O/T/N=1 root:Packages:NIST:Event:SplitFileWave
2872	WAVE tw = root:Packages:NIST:Event:SplitFileWave
2873
2874	List2TextWave(str,tw)
2875	Edit tw
2876
2877	return(0)
2878	End
2879
2880
2881	//// save the sliced data, and accumulate slices
2882	// *** this works with sliced data -- that is data that has been PROCESSED
2883	//
2884	// need some way of ensuring that the slices match up since I'm blindly adding them together.
2885	//
2886	// mode = 0 wipe out the old accumulated, copy slicedData to accumulatedData
2887	// mode = 1 add current slicedData to accumulatedData
2888	// mode = 2 copy accumulatedData to slicedData in preparation of export or display
2889	// mode = 3 unused...
2890	//
2891	// "Split Large File",SplitBigFile()
2892	// "Accumulate First Slice",AccumulateSlices(0)
2893	// "Add Current Slice",AccumulateSlices(1)
2894	// "Display Accumulated Slices",AccumulateSlices(2)
2895	//
2896	Function AccumulateSlicesButton(ctrlName) : ButtonControl
2897	String ctrlName
2898
2899	Variable mode
2900	mode = str2num(ctrlName[strlen(ctrlName)-1])
2901	// Print "mode=",mode
2902	AccumulateSlices(mode)
2903
2904	return(0)
2905	End
2906
2907	Function AccumulateSlices(mode)
2908	Variable mode
2909
2910	SetDataFolder root:Packages:NIST:Event:
2911
2912	switch(mode)
2913	case 0:
2914	DoAlert 0,"The current data has been copied to the accumulated set. You are now ready to add more data."
2915	KillWaves/Z accumulatedData
2916	Duplicate/O slicedData accumulatedData
2917	break
2918	case 1:
2919	DoAlert 0,"The current data has been added to the accumulated data. You can add more data."
2920	Wave acc=accumulatedData
2921	Wave cur=slicedData
2922	acc += cur
2923	break
2924	case 2:
2925	DoAlert 0,"The accumulated data is now the display data and is ready for display or export."
2926	Duplicate/O accumulatedData slicedData
2927	break
2928	default:
2929
2930	endswitch
2931
2932	SetDataFolder root:
2933	return(0)
2934	end
2935
2936
2937	////////////////////////////////////////////
2938	//
2939	// Panel and procedures for decimation
2940	//
2941	////////////////////////////////////////////
2942
2943	//Function E_ShowDecimateButton(ctrlName) : ButtonControl
2944	// String ctrlName
2945	//
2946	// DoWindow/F DecimatePanel
2947	// if(V_flag ==0)
2948	// Execute "DecimatePanel()"
2949	// endif
2950	//End
2951	//
2952	//
2953	//Proc DecimatePanel() //: Panel
2954	//
2955	// PauseUpdate; Silent 1 // building window...
2956	// NewPanel /W=(1602,44,1961,380)/K=1
2957	//// ShowTools/A
2958	// Button button0,pos={29,15},size={100,20},proc=SplitFileButtonProc,title="Split Big File"
2959	// SetVariable setvar0,pos={182,55},size={150,15},title="Decimation factor",fsize=10
2960	// SetVariable setvar0,limits={1,inf,1},value= root:Packages:NIST:Event:gDecimation
2961	// Button button1,pos={26,245},size={150,20},proc=LoadDecimateButtonProc,title="Load and Decimate"
2962	// Button button2,pos={25,277},size={150,20},proc=ConcatenateButtonProc,title="Concatenate"
2963	// Button button3,pos={25,305},size={150,20},proc=DisplayConcatenatedButtonProc,title="Display Concatenated"
2964	// Button button4,pos={29,52},size={130,20},proc=Stream_LoadDecim,title="Load From List"
2965	//
2966	// GroupBox group0 title="Manual Controls",size={185,112},pos={14,220}
2967	//EndMacro
2968
2969
2970	Function SplitFileButtonProc(ctrlName) : ButtonControl
2971	String ctrlName
2972
2973	Execute "SplitBigFile()"
2974	End
2975
2976
2977	// show all of the data
2978	//
2979	Proc ShowDecimatedGraph()
2980
2981	DoWindow/F DecimatedGraph
2982	if(V_flag == 0)
2983	PauseUpdate; Silent 1 // building window...
2984	String fldrSav0= GetDataFolder(1)
2985	SetDataFolder root:Packages:NIST:Event:
2986	Display /W=(25,44,486,356)/K=1/N=DecimatedGraph rescaledTime_dec
2987	SetDataFolder fldrSav0
2988	ModifyGraph mode=4
2989	ModifyGraph marker=19
2990	ModifyGraph rgb(rescaledTime_dec)=(0,0,0)
2991	ModifyGraph msize=1
2992	ErrorBars rescaledTime_dec OFF
2993	Label left "\\Z14Time (seconds)"
2994	Label bottom "\\Z14Event number"
2995	ShowInfo
2996	endif
2997
2998	EndMacro
2999
3000	// data has NOT been processed
3001	//
3002	// so work with x,y,t, and rescaled time
3003	// variables -- t_longest
3004	Function ConcatenateButtonProc(ctrlName) : ButtonControl
3005	String ctrlName
3006
3007	DoAlert 1,"Is this the first file?"
3008	Variable first = V_flag
3009
3010	fConcatenateButton(first)
3011
3012	return(0)
3013	End
3014
3015	Function fConcatenateButton(first)
3016	Variable first
3017
3018
3019	SetDataFolder root:Packages:NIST:Event:
3020
3021	Wave timePt_dTmp=timePt_dTmp
3022	Wave xLoc_dTmp=xLoc_dTmp
3023	Wave yLoc_dTmp=yLoc_dTmp
3024	Wave rescaledTime_dTmp=rescaledTime_dTmp
3025
3026	NVAR t_longest_dec = root:Packages:NIST:gEvent_t_longest_decimated
3027	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
3028
3029
3030	if(first==1) //1==yes, 2==no
3031	//then copy the files over, adjusting the time to start from zero
3032	// rescaledTime starts from zero (set by the loader)
3033
3034	timePt_dTmp -= timePt_dTmp[0] //subtract the first value
3035
3036	Duplicate/O timePt_dTmp timePt_dec
3037	Duplicate/O xLoc_dTmp xLoc_dec
3038	Duplicate/O yLoc_dTmp yLoc_dec
3039	Duplicate/O rescaledTime_dTmp rescaledTime_dec
3040
3041	t_longest_dec = t_longest
3042
3043	else
3044	// concatenate the files + adjust the time
3045	Wave timePt_dec=timePt_dec
3046	Wave xLoc_dec=xLoc_dec
3047	Wave yLoc_dec=yLoc_dec
3048	Wave rescaledTime_dec=rescaledTime_dec
3049
3050	// adjust the times -- assuming they add
3051	// rescaledTime starts from zero (set by the loader)
3052	//
3053	//
3054	rescaledTime_dTmp += rescaledTime_dec[numpnts(rescaledTime_dec)-1]
3055	rescaledTime_dTmp += abs(rescaledTime_dec[numpnts(rescaledTime_dec)-1] - rescaledTime_dec[numpnts(rescaledTime_dec)-2])
3056
3057	timePt_dTmp -= timePt_dTmp[0] //subtract the first value
3058
3059	timePt_dTmp += timePt_dec[numpnts(timePt_dec)-1] // offset by the last point
3060	timePt_dTmp += abs(timePt_dec[numpnts(timePt_dec)-1] - timePt_dec[numpnts(timePt_dec)-2]) // plus delta so there's not a flat step
3061
3062	Concatenate/NP/O {timePt_dec,timePt_dTmp}, tmp
3063	Duplicate/O tmp timePt_dec
3064
3065	Concatenate/NP/O {xLoc_dec,xLoc_dTmp}, tmp
3066	Duplicate/O tmp xLoc_dec
3067
3068	Concatenate/NP/O {yLoc_dec,yLoc_dTmp}, tmp
3069	Duplicate/O tmp yLoc_dec
3070
3071	Concatenate/NP/O {rescaledTime_dec,rescaledTime_dTmp}, tmp
3072	Duplicate/O tmp rescaledTime_dec
3073
3074
3075	KillWaves tmp
3076
3077	t_longest_dec = rescaledTime_dec[numpnts(rescaledTime_dec)-1]
3078
3079	endif
3080
3081
3082	SetDataFolder root:
3083
3084	return(0)
3085
3086	End
3087
3088	Function DisplayConcatenatedButtonProc(ctrlName) : ButtonControl
3089	String ctrlName
3090
3091	//copy the files over to the display set for processing
3092	SetDataFolder root:Packages:NIST:Event:
3093
3094	Wave timePt_dec=timePt_dec
3095	Wave xLoc_dec=xLoc_dec
3096	Wave yLoc_dec=yLoc_dec
3097	Wave rescaledTime_dec=rescaledTime_dec
3098
3099	NVAR t_longest_dec = root:Packages:NIST:gEvent_t_longest_decimated
3100	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
3101
3102	Duplicate/O timePt_dec timePt
3103	Duplicate/O xLoc_dec xLoc
3104	Duplicate/O yLoc_dec yLoc
3105	Duplicate/O rescaledTime_dec rescaledTime
3106
3107	t_longest = t_longest_dec
3108
3109	SetDataFolder root:
3110
3111	return(0)
3112
3113	End
3114
3115
3116
3117
3118	Function LoadDecimateButtonProc(ctrlName) : ButtonControl
3119	String ctrlName
3120
3121	LoadEventLog_Button("")
3122
3123	// now decimate
3124	SetDataFolder root:Packages:NIST:Event:
3125
3126	Wave timePt=timePt
3127	Wave xLoc=xLoc
3128	Wave yLoc=yLoc
3129	NVAR t_longest_dec = root:Packages:NIST:gEvent_t_longest_decimated
3130
3131	NVAR decimation = root:Packages:NIST:Event:gDecimation
3132
3133
3134	Duplicate/O timePt, timePt_dTmp
3135	Duplicate/O xLoc, xLoc_dTmp
3136	Duplicate/O yLoc, yLoc_dTmp
3137	Resample/DOWN=(decimation)/N=1 timePt_dTmp
3138	Resample/DOWN=(decimation)/N=1 xLoc_dTmp
3139	Resample/DOWN=(decimation)/N=1 yLoc_dTmp
3140
3141
3142	Duplicate/O timePt_dTmp rescaledTime_dTmp
3143	rescaledTime_dTmp = 1e-7*(timePt_dTmp - timePt_dTmp[0]) //convert to seconds and start from zero
3144	t_longest_dec = waveMax(rescaledTime_dTmp) //should be the last point
3145
3146	SetDataFolder root:
3147
3148
3149	End
3150
3151	// loads a list of files, decimating each chunk as it is read in
3152	//
3153	Function Stream_LoadDecim(ctrlName)
3154	String ctrlName
3155
3156	Variable fileref
3157
3158	SVAR filename = root:Packages:NIST:gEvent_logfile
3159	NVAR t_longest = root:Packages:NIST:gEvent_t_longest
3160
3161	SVAR listStr = root:Packages:NIST:Event:gSplitFileList
3162
3163	NVAR t_longest_dec = root:Packages:NIST:gEvent_t_longest_decimated
3164	NVAR decimation = root:Packages:NIST:Event:gDecimation
3165
3166
3167	// if "stream" mode is not checked - abort
3168	NVAR gEventModeRadioVal= root:Packages:NIST:gEvent_mode
3169	if(gEventModeRadioVal != MODE_STREAM)
3170	Abort "The mode must be 'Stream' to use this function"
3171	return(0)
3172	endif
3173
3174	// if the list has been edited, turn it into a list
3175	WAVE/T/Z tw = root:Packages:NIST:Event:SplitFileWave
3176	if(WaveExists(tw))
3177	listStr = TextWave2SemiList(tw)
3178	endif
3179
3180
3181
3182	//loop through everything in the list
3183	Variable num,ii
3184	num = ItemsInList(listStr)
3185
3186	for(ii=0;ii<num;ii+=1)
3187
3188	// (1) load the file
3189	filename = StringFromList(ii, listStr ,";")
3190
3191
3192	#if (exists("EventLoadWave")==4)
3193	LoadEvents_XOP()
3194	#else
3195	LoadEvents()
3196	#endif
3197
3198	SetDataFolder root:Packages:NIST:Event: //LoadEvents sets back to root:
3199
3200	Wave timePt=timePt
3201	Wave xLoc=xLoc
3202	Wave yLoc=yLoc
3203	CleanupTimes(xLoc,yLoc,timePt) //remove zeroes
3204
3205	Duplicate/O timePt rescaledTime
3206	rescaledTime = 1e-7*(timePt-timePt[0]) //convert to seconds and start from zero
3207	t_longest = waveMax(rescaledTime) //should be the last point
3208
3209	// (2) do the decimation, just on timePt. create rescaledTime from the decimated timePt
3210
3211	Duplicate/O timePt, timePt_dTmp
3212	Duplicate/O xLoc, xLoc_dTmp
3213	Duplicate/O yLoc, yLoc_dTmp
3214	Resample/DOWN=(decimation)/N=1 timePt_dTmp
3215	Resample/DOWN=(decimation)/N=1 xLoc_dTmp
3216	Resample/DOWN=(decimation)/N=1 yLoc_dTmp
3217
3218
3219	Duplicate/O timePt_dTmp rescaledTime_dTmp
3220	rescaledTime_dTmp = 1e-7*(timePt_dTmp - timePt_dTmp[0]) //convert to seconds and start from zero
3221	t_longest_dec = waveMax(rescaledTime_dTmp) //should be the last point
3222
3223
3224	// (3) concatenate
3225	fConcatenateButton(ii+1) //passes 1 for the first time, >1 each other time
3226
3227	endfor
3228
3229	//// Now that everything is decimated and concatenated, create the rescaled time wave
3230	// SetDataFolder root:Packages:NIST:Event: //LoadEvents sets back to root:
3231	// Wave timePt_dec = timePt_dec
3232	// Duplicate/O timePt_dec rescaledTime_dec
3233	// rescaledTime_dec = 1e-7*(timePt_dec - timePt_dec[0]) //convert to seconds and start from zero
3234	// t_longest_dec = waveMax(rescaledTime_dec) //should be the last point
3235
3236	DisplayConcatenatedButtonProc("")
3237
3238	SetDataFolder root:
3239
3240	return(0)
3241	End
3242
3243	/////////////////////////////////////

Note: See TracBrowser for help on using the repository browser.

Download in other formats:

Original Format