Update on ALICE software status and ideas Ervin

Update on ALICE software status and ideas Ervin Dénes Wigner Research Center Hungarian Academy of Sciences

How Does the RORC SW Work? • We have DMA engines in the RORC cards (FW)that take data over optical links (DDL) to store it in RAM. Detector DIU SIU RORC P c i PC DDL • We need a PCI driver for the RORC. • The driver maps the RORC’s registers into memory, so that the SW can read/write • The SW can give the addresses where to put the collected data and can assure the RORC always has addresses of free memory. E. Dénes, Wigner RC (Hu) Cross LHC exp. DAQ meeting, 9 July 2014 2

Physmem • To support the DDL transfers we need a RAM area which is pinned (non-swapped), with known physical addresses, and which is accessible from user-level code. • For that at boot time we limit the amount of memory used by Linux (grub parameter mem=. . . ) e. g. : kernel /vmlinuz-2. 6. 32 -358. 23. 2. el 6. x 86_64 …. . nopat mem=4 G Physmem area 0 ………… Physmem area #n Linux RAM ~2, 5 G 4 G • We have a small device driver that maps the memory outside the Linux area via remap_pfn_range() • The driver gives to the user SW the size of the mapped area and its physical address E. Dénes, Wigner RC (Hu) Cross LHC exp. DAQ meeting, 9 July 2014 3

Evolution of Physmem Driver • Around year 2000 we started with Bigphys – Part of the Linux kernel – Not supported by IT --> for every new kernel version we had to recompile the kernel • First Physmem from 2002 – Kernel modul: no need of kernel compilation • Modifications for SLC 3 and SLC 4 (2006, 2008) • Compatible 32 and 64 bit versions for SLC 5 (2008) – Solved to skip the “hole” under 4 G and 4 G crosses 1 physmem area • Modifications for SLC 6 (2012) – /proc/iomem changed, does not report above boot parameter -> we use BIOS map • Kernel 2. 6. 32 uses new memory management – using PAT (Page Attribute Table) which is not correct for mapping big memory (> 3. 8 GB): read and write speeds fall drastically. Solution: “nopat” in GRUB (2013) • Modifications for Ubuntu (for C-RORC tests) (2013) E. Dénes, Wigner RC (Hu) Cross LHC exp. DAQ meeting, 9 July 2014 4

Evolution of RORC Driver • First driver for PCI 32 RORC: 2001 • 2001 – 2012: – following the evolution of SLC 2, 3, 4, 5, 6 kernels – following the evolution of RORC HW: • 66 MHz PCI, 100 MHz PC-X, PCI-Express Gen 2 • 2013: – Common driver for D-RORC and C-RORC – modifications for Ubuntu • Ubuntu is needed for HW tests of C-RORC only • RUN 3: – Commercial 10/40 Gb. E or PCIe over fibre HW come with their own drivers – Performance should be checked E. Dénes, Wigner RC (Hu) Cross LHC exp. DAQ meeting, 9 July 2014 5

How to follow? • Should we follow with Physmem? – Pros: • • no data move between kernel and user space no limit of memory size several process can reach data in the same time any part of the memory can be selected e. g. use this – Cons: • Linux developers do not support • What else could be used? – Scatter-Gather DMA is one option • Pros: – No need to develop memory management driver • Cons: – Not known if sg. DMA has all features what Physmem has E. Dénes, Wigner RC (Hu) Cross LHC exp. DAQ meeting, 9 July 2014 6

Next steps in common • There is some opportunity for – Exchange of information – Common development E. Dénes, Wigner RC (Hu) Cross LHC exp. DAQ meeting, 9 July 2014 7