Di FX Overview Adam Deller NRAO 3 rd
![Di. FX Overview Adam Deller NRAO 3 rd Di. FX workshop, Curtin University, Perth Di. FX Overview Adam Deller NRAO 3 rd Di. FX workshop, Curtin University, Perth](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-1.jpg)
![Outline l l SVN layout mpifxcorr: The heart of Di. FX l l l Outline l l SVN layout mpifxcorr: The heart of Di. FX l l l](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-2.jpg)
![SVN layout l Two ways to get what you need from SVN l l SVN layout l Two ways to get what you need from SVN l l](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-3.jpg)
![Mpifxcorr architecture Data. Stream 1 Baseband data processing buffer Core 1 processing buffer Data. Mpifxcorr architecture Data. Stream 1 Baseband data processing buffer Core 1 processing buffer Data.](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-4.jpg)
![Fx. Manager correlation flow l l Start at the requested time, step one block Fx. Manager correlation flow l l Start at the requested time, step one block](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-5.jpg)
![Datastream correlation flow l l l Two threads: Main (receives instructions, sends data) and Datastream correlation flow l l l Two threads: Main (receives instructions, sends data) and](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-6.jpg)
![Datastream correlation flow • Start time • Valid samples • Num sent • MPI_Send Datastream correlation flow • Start time • Valid samples • Num sent • MPI_Send](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-7.jpg)
![Core correlation flow l l N+1 threads: 1 for send/receive, the rest to do Core correlation flow l l N+1 threads: 1 for send/receive, the rest to do](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-8.jpg)
![Under the hood in Core l l l Each thread is identical, and has Under the hood in Core l l l Each thread is identical, and has](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-9.jpg)
![Core in pictures Baseband data from each telescope Read/send thread Subint visibilities Subint slot Core in pictures Baseband data from each telescope Read/send thread Subint visibilities Subint slot](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-10.jpg)
![Scaling and writing visibilities l l l Along with the sub-integrated visibilities, the Core Scaling and writing visibilities l l l Along with the sub-integrated visibilities, the Core](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-11.jpg)
![vex 2 difx: control file generator l l l Walter will speak in much vex 2 difx: control file generator l l l Walter will speak in much](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-12.jpg)
![. input file tables l l l Common: other files, start/stop etc Configuration: Number . input file tables l l l Common: other files, start/stop etc Configuration: Number](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-13.jpg)
![. calc file setup l l l Not explicitly tables like the. input file . calc file setup l l l Not explicitly tables like the. input file](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-14.jpg)
![calcif 2: geometric model gen. l l calcif 2 takes the info given in calcif 2: geometric model gen. l l calcif 2 takes the info given in](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-15.jpg)
![difx 2 fits: FITS-IDI builder l l The visibilities written out of the correlator difx 2 fits: FITS-IDI builder l l The visibilities written out of the correlator](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-16.jpg)
![The thorny scaling problem l l At least 5 different amplitude effects need to The thorny scaling problem l l At least 5 different amplitude effects need to](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-17.jpg)
![Recap: correlation flow chart In: nothing Out: Generate. v 2 d file . v Recap: correlation flow chart In: nothing Out: Generate. v 2 d file . v](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-18.jpg)
- Slides: 18
![Di FX Overview Adam Deller NRAO 3 rd Di FX workshop Curtin University Perth Di. FX Overview Adam Deller NRAO 3 rd Di. FX workshop, Curtin University, Perth](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-1.jpg)
Di. FX Overview Adam Deller NRAO 3 rd Di. FX workshop, Curtin University, Perth
![Outline l l SVN layout mpifxcorr The heart of Di FX l l l Outline l l SVN layout mpifxcorr: The heart of Di. FX l l l](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-2.jpg)
Outline l l SVN layout mpifxcorr: The heart of Di. FX l l l Data management and flow The “Core” of the mpifxcorr Scaling and writing of visibilities I will focus on this segment of Di. FX Briefly, the surrounding infrastructure: l l l vex 2 difx: input/calc file generator calcif 2: Geometric model generator difx 2 fits: FITS builder Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![SVN layout l Two ways to get what you need from SVN l l SVN layout l Two ways to get what you need from SVN l l](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-3.jpg)
SVN layout l Two ways to get what you need from SVN l l Tagged versions: these live under “master_tags” at the top level. Current version is Di. FX-1. 5. 1 Active versions: branches/difx-1. 5, trunk==difx 2. 0 Tagged versions: frozen. Recommended. Flat structure ie master_tags/mpifxcorr/*, master_tags/vexdifx/*, … Active versions: Can change! Structure includes “branch” underneath: applications/vex 2 difx/branches/difx-1. 5/*, … Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![Mpifxcorr architecture Data Stream 1 Baseband data processing buffer Core 1 processing buffer Data Mpifxcorr architecture Data. Stream 1 Baseband data processing buffer Core 1 processing buffer Data.](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-4.jpg)
Mpifxcorr architecture Data. Stream 1 Baseband data processing buffer Core 1 processing buffer Data. Stream 2 Source data … Core 2 … Data. Stream N Core M Large, segmented ring buffer Up to 100 s MB/ a few or more seconds Timerange, destination Visibilities Master Node Visibility buffer MPI is used for inter-process communications Each data transfer is double buffered Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![Fx Manager correlation flow l l Start at the requested time step one block Fx. Manager correlation flow l l Start at the requested time, step one block](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-5.jpg)
Fx. Manager correlation flow l l Start at the requested time, step one block of FFTs at a time until end of correlation Di. FX 1. 5 implementation: Contiguous time. Skip if time has no active “Configuration” Di. FX 2. 0 implementation: correlate one scan at a time - whole scan either matches Configuration or not (contiguous not required) As visibilities are completed, release lock on visibility buffer slot (second thread writes out) Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![Datastream correlation flow l l l Two threads Main receives instructions sends data and Datastream correlation flow l l l Two threads: Main (receives instructions, sends data) and](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-6.jpg)
Datastream correlation flow l l l Two threads: Main (receives instructions, sends data) and read (fills the buffer) Each maintains a lock on at least one segment of the databuffer at all times While data remains, the read thread will keep populating the data buffer until told to stop Main thread just dumbly fulfils requests until told to stop by Manager Sends a short flag to Core if no valid data Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![Datastream correlation flow Start time Valid samples Num sent MPISend Datastream correlation flow • Start time • Valid samples • Num sent • MPI_Send](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-7.jpg)
Datastream correlation flow • Start time • Valid samples • Num sent • MPI_Send * handle • Lock Read thread Data buffer Requested time sent to Core “Segment” “Send” Send thread FFT = 2 x num channels …. . Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![Core correlation flow l l N1 threads 1 for sendreceive the rest to do Core correlation flow l l N+1 threads: 1 for send/receive, the rest to do](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-8.jpg)
Core correlation flow l l N+1 threads: 1 for send/receive, the rest to do actual correlation (. threads file) One buffer slot is processed at a time - each process thread gets 1/Nth of the FFTs More locking is required so the threads can aggregate their results, which are stored in one long array (for ease of sending back) Keeps looping until a terminate message is received from Fx. Manager Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![Under the hood in Core l l l Each thread is identical and has Under the hood in Core l l l Each thread is identical, and has](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-9.jpg)
Under the hood in Core l l l Each thread is identical, and has an array of “Mode” objects, which handle the stationbased processing for each Datastream Mode knows how to unpack the different formats, and then handles fringe rotation, FFT and fractional sample correction After telling each Mode to do its thing, the thread grabs the appropriate results and XMACs, as described in the input file Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![Core in pictures Baseband data from each telescope Readsend thread Subint visibilities Subint slot Core in pictures Baseband data from each telescope Read/send thread Subint visibilities Subint slot](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-10.jpg)
Core in pictures Baseband data from each telescope Read/send thread Subint visibilities Subint slot Mode objects for each datastream Core object Baseband data pointer unpacked data Intermediatiate data Proc. thread XMAC Thread visibilities Adam Deller Final data for XMAC Repeated for each subband 3 rd Di. FX workshop, Curtin University, Perth
![Scaling and writing visibilities l l l Along with the subintegrated visibilities the Core Scaling and writing visibilities l l l Along with the sub-integrated visibilities, the Core](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-11.jpg)
Scaling and writing visibilities l l l Along with the sub-integrated visibilities, the Core maintains a count of valid samples that were used, also sends to Fx. Manager This is divided by the expected number of samples and used to adjust the visibility amplitude up (and weight down) at the Fx. Manager, before writing to disk At the same time, visibilities can be scaled by mean autocorrelations and predicted Tsys, but this is not recommended with difx 2 fits Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![vex 2 difx control file generator l l l Walter will speak in much vex 2 difx: control file generator l l l Walter will speak in much](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-12.jpg)
vex 2 difx: control file generator l l l Walter will speak in much more detail on this For the running of Di. FX, its two important outputs are the. input file (which describes the setup of the correlator) and the. calc file (which describes the antenna positions, scan durations, source coords etc) Input. v 2 d file must provide vex file that describes observation; can also override defaults for numchannels, int time, … Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![input file tables l l l Common other files startstop etc Configuration Number . input file tables l l l Common: other files, start/stop etc Configuration: Number](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-13.jpg)
. input file tables l l l Common: other files, start/stop etc Configuration: Number of channels, anything that might want to change from one scan to the next Freq: IF frequencies, bandwidth, sideband Datastream: Setup for each telescope Baseline: What bands to correlate Data/network: Where to load the data from Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![calc file setup l l l Not explicitly tables like the input file . calc file setup l l l Not explicitly tables like the. input file](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-14.jpg)
. calc file setup l l l Not explicitly tables like the. input file Start/stop time (not necessarily same as. input file Antenna info (xyz, mount type, axis offset, …) Source info (RA, dec, [parallax/pm…]) Scan info (start, duration, source) Earth Orientation Parameters (EOPs) Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![calcif 2 geometric model gen l l calcif 2 takes the info given in calcif 2: geometric model gen. l l calcif 2 takes the info given in](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-15.jpg)
calcif 2: geometric model gen. l l calcif 2 takes the info given in the. calc file and produces predicted delays and uvws for each datastream This is stored in two forms: l l A densely packed series of samples (usually every second). This is the. delay and. uvw file used by Di. FX 1. 5, which interpolates the samples A less densely sampled set of polynomials (usually every 2 minutes). This is the. im file used by Di. FX 2. 0 - it’s a bit smaller/more efficient Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![difx 2 fits FITSIDI builder l l The visibilities written out of the correlator difx 2 fits: FITS-IDI builder l l The visibilities written out of the correlator](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-16.jpg)
difx 2 fits: FITS-IDI builder l l The visibilities written out of the correlator are in a very simple format: ascii header, followed by 32 bit float binary data difx 2 fits takes these visibilities and the metadata from the control files, and builds a FITS-IDI file that can be loaded into eg AIPS More than one correlation can be combined into a single FITS file if compatible Further scaling of amplitudes is done Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![The thorny scaling problem l l At least 5 different amplitude effects need to The thorny scaling problem l l At least 5 different amplitude effects need to](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-17.jpg)
The thorny scaling problem l l At least 5 different amplitude effects need to be corrected (van Vleck, unpack vals, …) FITLD in AIPS is hardcoded to do some of these Therefore the combination of what is done at Fx. Manager + difx 2 fits must match what FITLD expects Further discussion of this in the Tuesday afternoon session, hopefully Adam Deller 3 rd Di. FX workshop, Curtin University, Perth
![Recap correlation flow chart In nothing Out Generate v 2 d file v Recap: correlation flow chart In: nothing Out: Generate. v 2 d file . v](https://slidetodoc.com/presentation_image/6f14670ae6e3ad0256a103775a77858d/image-18.jpg)
Recap: correlation flow chart In: nothing Out: Generate. v 2 d file . v 2 d vex 2 difx . calcif 2 ? ? local info, . input Generate. threads/machine file . input, . calc. . delay/uvw/im . input/delay/uvw machines, . threads Correlate: mpifxcorr . difx, . input, . im Difx 2 fits Adam Deller . v 2 d . threads/machines. difx (visibilities) FITS file 3 rd Di. FX workshop, Curtin University, Perth
Adam deller
Tweaking
Deller
Nrao
Nrao
Mynrao
Nrao
Nrao
Adam adam facebook
Toefl
Methodologies for cross-domain data fusion: an overview
Areanna ai
Total atp produced in cellular respiration
Walmart company profile
Avamar ndmp accelerator
Indian banking sector overview
Overview of http
Overview button
Overview of oracle architecture