G A P Cirrone S Donadio S Guatelli

G. A. P. Cirrone, S. Donadio, S. Guatelli, A. Mantero, B. Mascialino, S. Parlati, A. Pfeiffer, M. G. Pia, A. Ribon, P. Viarengo 9 th Topical Seminar on Innovative Particle and Radiation Detectors 23 - 26 May 2004 Siena, Italy Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Data analysis in HEP Provide tools for the statistical comparison of distributions in terms of: v Equivalent reference distributions; v Experimental measurements; v. Data v from reference sources; Functions deriving from theoretical calculations or fits; Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Applications v Validation of Geant 4 v. Detector monitoring; v. Simulation validation; v Reconstruction vs. Expectation; v Regression testing; v Physics analysis; analysis Stefania Donadio electromagnetic physics models v Attenuation coefficients, CSDA ranges, Stopping Power, distributions of physics quantities v Quantitative comparisons to experimental data and recognised standard references 9 th TSIPRD Siena, May 23 -26 2004

Example of Applications I Photon mass attenuation coefficient NIST G 4 Standard G 4 Low. E Photon beam (Io) Transmitted photons (I) Absorber Materials: Materials Be, Al, Si, Ge, Fe, Cs, Au, Pb, U Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Example of Applications II Electron stopping power and CSDA range Absorber Materials: Materials Be, Al, Si, Ge, Fe, Cs, Au, Pb, U Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Go. F statistical toolkit Quantitative evaluation Qualitative evaluation A project to develop a statistical comparison system Comparison of distributions Goodness of fit testing Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Software Process guidelines • United Software Development Process, specifically tailored to the project – practical guidance and tools from the RUP – both rigorous and lightweight – mapping onto ISO 15504 • Guidance from ISO 15504 • Incremental and iterative life cycle model with SPIRAL APPROACH Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Architectural guidelines • The project adopts a solid architectural approach – – – • • to offer the functionality and the quality needed by the users to be maintainable over a large time scale to be extensible, to accommodate future evolutions of the requirements Component-based approach – to facilitate re-use and integration in different frameworks AIDA – adopt a (HEP) standard – no dependence on any specific analysis tool Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

The algorithms are specialised on the kind of distribution (binned/unbinned) Every algorithm has been rigorously tested Documentation available : http: //www. ge. infn. it/geant 4/analysis/HEPstatistics/ Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Chi-Squared test • Applies to binned distributions • It can be useful also in case of unbinned distributions, but the data must be grouped into classes • Cannot be applied if the counting of theoretical frequencies in each class is < 5 – When this is not the case, one could try to unify contiguous classes until the minimum theoretical frequency is reached – Otherwise one could use Yates formula Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

More sophisticated algorithms unbinned distributions • Kolmogorov-Smirnov test • Goodman approximation of KS test • Kuiper test EMPIRICAL DISTRIBUTION FUNCTION ORIGINAL DISTRIBUTIONS Dmn Stefania Donadio 9 th TSIPRD SUPREMUM STATISTICS Siena, May 23 -26 2004

More powerful algorithms unbinned distributions • Cramer-von Mises test • (Tiku test) • Anderson-Darling test TESTS CONTAINING A WEIGHTING FUNCTION These algorithms are so powerful that we decided to implement their equivalent in case of binned distributions: binned distributions • Fisz-Cramer-von Mises test • (Tiku test) • k-sample Anderson-Darling test Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

How to decide the power of an algorithm? A test is considered powerful if the probability of accepting the null hypothesis when null hypothesis is wrong is low 2 < Supremum statistics tests < Tests containing a weight function v 2 loses information in a test for unbinned distribution by grouping the data into cells (Kac, Kiefer and Wolfowitz (1955) showed that Kolmogorov-Smirnov test requires n 4/5 observations compared to n observations for 2 to attain the same power) v Cramer-von Mises and Anderson-Darling statistics are expected to be superior to Kolmogorov-Smirnov’s, since they make a comparison of the two distributions all along the range of x, rather than looking for a marked difference at one point. . This is now work in progress. . . Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

User’s point of view • Simple user layer • Only deal with AIDA objects and choice of comparison algorithm The user is completely shielded from both statistical and computing complexity. USER TOOLKIT STATISTICAL RESULT EXTRACTS THE ALGORITHM WRITING ONE LINE OF CODE Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Results and practical applications Collaborations with: Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Microscopic validation of physics NIST 2 N-S=0. 267 =28 p=1 Geant 4 Standard 2 N-L=1. 315 =28 p=1 Geant 4 Low. E Chi-squared test 2 N-S=0. 373 =28 p=1 Geant 4 simulations are statistically comparable with reference data (NIST database =0. 532 =28 p=1 2 N-S=0. 532 =28 p=1 http: //www. nist. gov) 2 2 N-L= 5. 882 =28 p=1 N-S 2 N-L=1. 928 =28 p=1 2 Stefania Donadio N-L=1. 928 9 th TSIPRD =28 p=1 Siena, May 23 -26 2004

Test beam at Bessy Bepi-Colombo Mission X-ray fluorescence spectrum in Iceand basalt (EIN=6. 5 ke. V) Very complex distributions Counts Experimental measurements are comparable with Geant 4 simulations A. Mantero, M. Bavdaz, A. Owens, A. Peacock, M. G. Pia Simulation of X-ray Fluorescence and(ke. V) Application to Planetary Energy Astrophysics Stefania Donadio 9 th TSIPRD Chi 2 not appropriate (< 5 entries in some bins, physical information would be lost if rebinned) Anderson-Darling Ac (95%) =0. 752 Siena, May 23 -26 2004

Medical applications in hadron therapy KOLMOGOROV-SMIRNOV Experimental measurements are comparable with Geant 4 simulations DEXP-GEANT 4=0. 11 p=n. s. Goodman approximation KOLMOGOROV-SMIRNOV 2 EXP-GEANT 4=3. 8 =2 p=n. s. G. A. P. Cirrone, G. Cuttone, S. Donadio, S. Guatelli, S. Lo Nigro, B. Mascialino, M. G. Pia, L. Raffaele, G. M. Sabini Implementation of a new Monte Carlo Simulation Tool for the Development of a proton Therapy Beam Line and Verification of the Related Dose Distributions Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004

Conclusions • This is a new up-to-date easy to handle and powerful tool for statistical comparison in particle physics. • It the first tool supplying such a variety of sophisticated and powerful statistical tests in HEP. • AIDA interfaces allow its integration in any other data analysis tool. Applications in: HEP, astrophysics, medical physics Stefania Donadio 9 th TSIPRD Siena, May 23 -26 2004