Software for MAPS and MERLIN T G Perring

  • Slides: 19
Download presentation
Software for MAPS and MERLIN T. G. Perring ISIS Facility, Rutherford Appleton Laboratory

Software for MAPS and MERLIN T. G. Perring ISIS Facility, Rutherford Appleton Laboratory

Outline Plan of talk: • Overview • Brief description of the present MAPS/Het/MARI software

Outline Plan of talk: • Overview • Brief description of the present MAPS/Het/MARI software • Challenges of MERLIN/ARCS • How we plan to re-write the software Our aims from this meeting: • Contribute to the plans for direction of ARCS software development • Explore cooperation in software development – Experience + a working ARCS-like spectrometer – Fresh view of the problems of handling huge inelastic datasets

MAPS spectrometer Specification: Background chopper Monochromating chopper • 20 me. V< EI< 2000 me.

MAPS spectrometer Specification: Background chopper Monochromating chopper • 20 me. V< EI< 2000 me. V • lmod-chop = 10 m Sample position • lsam-det = 6 m Position sensitive detector array • low angle bank: 3 -20 high angle bank: 60 • hbar /EI = 1 - 5% (FWHH) • 40, 000 detector elements 2500 time channels 108 pixels 0. 4 GB datasets

Overview Inelastic scattering from single crystals: • Triple axis spectrometer: point-by-point serial operation –

Overview Inelastic scattering from single crystals: • Triple axis spectrometer: point-by-point serial operation – Few k. B of data – Every group of users writes its own least-squares fitting algorithm from scratch – Ad-hoc cooperation only MAPS/MERLIN/ARCS: • Parallel operation: collecting on 3 D surface in 4 D S(Q, )-space – MAPS: raw data: 108 pixels = 0. 4 GB corrected data: 1 -2 x 107 = 0. 1 -0. 2 GB MERLIN, ARCS: 1 -2 x 107 = 0. 1 -0. 2 GB x x 10 datasets 100+ datasets • Need ‘Rietveld refinement’ for inelastic scattering – Groups cannot afford to write own software – Pointless anyway: e. g. in diffraction GSAS, CCSL are trusted – We should move to a similar model of trusted black boxes

Overview (Rietveld refinement cont. ) • Use entire data set to extract a limited

Overview (Rietveld refinement cont. ) • Use entire data set to extract a limited number of parameters in a model – Magnetic coupling constants – Force constants – Full inversion of the data: if S(Q, ) in 4 D • Pragmatism: – – In magnetism S(Q, ) (or ''(Q, )) is the quantity to test Qualitative features (are there antiferromagnetic fluctuations and if so, where ? ) Unknown processes distorting measure of goodness of fit Rapid slicing and dicing, testing models on limited volumes of data, testing implications for other parts of the data volume, feeding back into operation of experiment seamless integration of simulation, visualisation and analysis programs More formal cooperation: • Agreed data structures for instrument information, counts/ S(Q, ) • Definitions for input/output of algorithms • Capitalise on the investment we individually make in algorithms

Current software on MAPS, HET, MARI Planning experiment Monitoring experiment Data reduction Tobyplot Genie-II

Current software on MAPS, HET, MARI Planning experiment Monitoring experiment Data reduction Tobyplot Genie-II HOMER (reciprocal space viewing) Open. Genie-II Chop (resolution, flux) Visualization Analysis MSLICE Tobyfit (single crystals) GUI driven Matlab [Ad-hoc programs in Multi-frills MFIT MSCATT]

Tobyplot Ei=450 me. V Psi=101. 3° • Reciprocal space viewing • Mostly important for

Tobyplot Ei=450 me. V Psi=101. 3° • Reciprocal space viewing • Mostly important for 3 D systems • but very useful for 1 D, 2 D, to assess access in reciprocal space Fortran 77 + PGPLOT VMS/Windows/Unix

CHOP • Flux • Resolution: • At elastic position • As function of Test

CHOP • Flux • Resolution: • At elastic position • As function of Test flux/resolution compromises Fortran 77 + PGPLOT VMS/Windows/Unix

Monitoring an Experiment GENIE-II I(t) for single spectrum rebinning, units conversion, integration Algebra on

Monitoring an Experiment GENIE-II I(t) for single spectrum rebinning, units conversion, integration Algebra on spectra [e. g. W 1=(0. 3 W 2 + W 3)/W 4] can call user-written FORTRAN algorithms Good for quick checks during and after experiments Still use today on MAPS VMS only Open. GENIE New generation Used on many ISIS instruments Windows, Unix, VMS Not the features of Matlab, IDL Is free, however

Data reduction HOMER (au. Ray Osborn) I(detno, t) corrects Kf/ki, efficiency(kf) Map file (detectors

Data reduction HOMER (au. Ray Osborn) I(detno, t) corrects Kf/ki, efficiency(kf) Map file (detectors workspaces) DIAG ( list of bad detectors) S(detno, ) ASCII output file (or VMS binary) MONO_VAN ( absolute units conversion) WHITE_VAN ( solid angle of detectors) • Encapsulates years of experience of the instruments • Scaled very well even to MAPS $ homer/map=par: pix_981. map/mask=8900/van=8850 8900 100 -30 95 0. 25

MSLICE: GUI interface Imaging single crystal data on HET, MARI, MAPS, and IRIS (Radu

MSLICE: GUI interface Imaging single crystal data on HET, MARI, MAPS, and IRIS (Radu Coldea (ISIS / Oak Ridge now Oxford) 2 D slices in (Q, ) ·run info. ·sample parameters · 2 D & 1 D cuts in (Q, )

TOBYFIT : Least squares fitting of resolution broadened cross-section models (Toby Perring, ISIS) Simultaneous

TOBYFIT : Least squares fitting of resolution broadened cross-section models (Toby Perring, ISIS) Simultaneous fitting to many 2 D or 1 D data sets Text-based interface for entering: ·instrument ·sample parameters ·cross-section parameters

MSLICE and TOBYFIT Integral part of the operation of the spectrometer - ‘Tertiary spectrometer’

MSLICE and TOBYFIT Integral part of the operation of the spectrometer - ‘Tertiary spectrometer’ MSLICE: Visualisation of 3 D data in 2 D slices, 1 D cuts Can generate ‘backgrounds’ from selected parts of the data MATLAB as front end: GUI Graphics manipulating data structures, ad-hoc programming FORTRAN 77 for speed of operation of algorithms PC with 1 GB RAM, 500 MHz+ necessary TOBYFIT: Fitting and simulation test ideas by feeding S(detno, ) to MSLICE Hold an experiment in a parameter file Fortran + PGPLOT Runs on VMS, Windows, UNIX Communicate via ASCII files: 1 for data, 1 for detector parameters other sample & instrument parameters

Challenges offered by future instrumentation • Physics a function of 4 variables: C(x 1,

Challenges offered by future instrumentation • Physics a function of 4 variables: C(x 1, x 2, x 3, x 4) • Instrument gathers data on a 3 D volume in that 4 D space • Data gathered on a fine non-Cartesian grid • 0. 1 -0. 2 Gbyte • MAPS: usually ~10 settings in an experiment • fine data on 3 D surface, coarse in 4 th dimension • ARCS/MERLIN: 100+ settings • fine in all 4 dimensions (scan Ei, or crystal orientation) • 10 -20 GByte complete data set • Data has low statistics - need techniques to pick out features in data • will always need real-time slicing and dicing of data [too many ways of being led astray or being deceived] • will be doing this after going back to home institution

Visualisation: a hierarchy of views • View 4 D data: • ? How do

Visualisation: a hierarchy of views • View 4 D data: • ? How do that ? • Define integration interval along any one dimension, and then: • View 3 D data: • isosurfaces + slider control for • intensity levels • rotation and viewpoint • binning along the three axes • smoothing, image processing control • Move a plane through the 3 D volume: define integration interval along one of the remaining dimensions, and then: • View 2 D data: • Contour plots, mountain plots, + slider controls for • contouring levels • interval scaling (linear, log, sqrt …) • binning along the two axes • Move a line across the plane, defining a thickness, and then • View 1 D data: • overplotting, fine comparison • book-keeping of titles of the plots. . . On raw counts, white beam files, … as well as S(Q, )

Instrument resolution, modelling • Must be able to simulate results of experiments and view

Instrument resolution, modelling • Must be able to simulate results of experiments and view results in same way as data (number crunching) • Must perform on-line analysis (resolution-convoluted model fitting, multiple scattering) within framework of same package (even more number crunching) • flexibility: fit on limited volume of data, simulate for whole dataset, sliceand-dice in same way as data to try out ideas • User wants one-stop shop I(det, t) Visualisation, Compare, fit Sexp(Q, w) * inst S calc(Q, w) Convolve with instrument algebra on 1, 2, 3, 4 D + Tobyplot Mk. II

Issues • Number crunching - more than the typical user institute will have •

Issues • Number crunching - more than the typical user institute will have • huge storage requirements (not just the raw data) • data management a real problem - we already create hundreds of cuts • thumbnails when click on file • database functions (select by date, temperature, field, scan of a parameter…) • history of analysis stored in file • well-defined data structures needed • ease interact seamlessly with other programs • deconvolution, modelling • user-written algorithms easy to write • define appropriate methods and algebras • (addition, subtraction, background generation, symmetrization…) • not just GUIs: • scripting must always be possible • maximises flexibility

Our plans Existing programs need to rewritten • F 77, getting unmaintainable, monolithic, functionality

Our plans Existing programs need to rewritten • F 77, getting unmaintainable, monolithic, functionality insufficient, grown organically, written independently • Define Ne. Xus files to hold all processed files • 1 D, 2 D, 3 D, 4 D data + all relevant instrument information (detectors etc. ) • sufficient information for MCSTAS simulation • include raw data files eventually • Mirror the data structures in Fortran 95 + methods • Fortran 95 for speedy algorithms [number crunching for visualisation (MSLICE-2), fitting (Tobyfit-2)] • MATLAB for graphics, language for manipulation and scripting - the glue [+access to all features of MATLAB for ad-hoc manipulations] TGP, SMB + inst. Scientists + part effort from 2 post-docs (25 -50% effort from each)

(plans cont. ) About to start working with E-Science centre at Rutherford Laboratory •

(plans cont. ) About to start working with E-Science centre at Rutherford Laboratory • funding to implement grid based applications for science within the laboratory • demonstration projects: • data portal to distributed data stores • graphics processing and number-crunching • ISIS projects chosen to focus the development for real applications • aim to isolate user from where the work is done - no need for their own Beowulf cluster • user will have a front end - in our case we want MATLAB but also web-based interface (oceanography, space science) • Ensures that one version is maintained • assumes high-speed networks • [GLOBUS toolkit to isolate user from location of resources] • ISIS: full effort of ISIS computing staff member + post-doc effort of E-Science centre