SLAC Stanford Linear Accelerator Center FECCIII INTEGRATION ISSUES

SLAC Stanford Linear Accelerator Center FECC-III INTEGRATION ISSUES Eric J. Siskind June 12, 2003 E. Siskind, June 12, 2003

SLAC Project Goals • Replace Multibus-I micro hardware with commercial-off-the-shelf personal computers. • Replace special purpose accelerator networks (SLCnet, KISnet, PNET) with a single commercial-off-the-shelf TCP/IP network such as switched gigabit Ethernet. • Develop personal computer CAMAC interface to replace MBCD/MBCD-II. E. Siskind, June 12, 2003 2

SLAC New CAMAC Features • CAMAC interface differs from MBCD/MBCD-II in four major ways: – Two board design to support remote I/O; – Real-time preemptive hardware with recovery; – Maximizes CAMAC utilization when user software environment in PC is multi-threaded; – Large (i. e. comparable to existing micro) user programming environment within interface. E. Siskind, June 12, 2003 3

SLAC Hardware Architecture • Hardware consists of two circuit boards connected by up to 10 km of single mode fiber optics: – PCIL (“PCI Link”) board plugs into PCI option slot in PC; – FECC (“Front-End CAMAC Controller”) is now a stand-alone chassis with a single-width CAMAC module to access PDU triggers. • PC moves from micro’s current location to MCC computer room, leaving behind only FECC. E. Siskind, June 12, 2003 4

SLAC PCIL Features • Basically an intelligent special-purpose Network Interface Card. • Processor with DMA PCI block transfer master and DMA fiber optic link transmitter/receiver. • Makes no assumptions about nature of PC’s realtime operating system. • Semi-static assembly language PROM code with downloaded updates. • First generation hardware doubles as Alpha SLCnet interface. E. Siskind, June 12, 2003 5

SLAC PCIL Code Functions • Supports PC FECC message passing. • Supports FECC remote read/write access to PC memory with optional PC interrupt. • Performs function-specific scatter/gather DMA to/from PC memory for all classes of operations, avoiding unnecessary buffer copying by PC CPU. • Forwards trigger pattern from PCI registers to FECC (only in 1 st generation; pattern forwarding in hardware in 2 nd generation). E. Siskind, June 12, 2003 6

SLAC FECC Features • Logical successor to MBCD/MBCD-II. • Accesses host memory via point-to-point fiber optic link instead of parallel I/O bus. • 1 st generation supports MBCD-II’s four strings of SLAC serial CAMAC crate controllers (5 MHz, bit serial, half duplex). • 2 nd generation adds support for four strings of IEEE standard crate controllers (5 MHz, byte serial, full duplex) and dual Bitbus strings. E. Siskind, June 12, 2003 7

SLAC CAMAC Interrupt Recovery & Control • If a CAMAC block transfer is interrupted, hardware can repeat the previous write operation (address pointer load? ) with updated write data before resuming the transfer. • Hardware can prevent an interrupt between CAMAC cycles transferring two 16 -bit halves of a single 32 -bit word. • Hardware can prevent an interrupt of a particular block transfer entirely (needed because of a current PIOP feature). E. Siskind, June 12, 2003 8

SLAC System Performance • Many PC jobs now can have simultaneously outstanding long CAMAC packages. • All packages are sent to FECC firmware ASAP, and parsed into their individual (block transfer) operations in a single crate (“packets”). • Packets are immediately queued to hardware for the cable accessing the target crate. • Cable operation begins as soon as any outstanding package has a packet requiring access to a crate on that cable. • Parallelism now achieved over multiple packages. E. Siskind, June 12, 2003 9

SLAC PCIL 2 Intelligence • Analog Devices ADSP-21060 “SHARC” 40 MHz 32 -bit DSP on 2 nd generation PCIL. • 40 k x 48 bit on-chip program memory; 64 k x 32 bit on-chip data memory. • 256 k x 48 bit external program/data memory. • 256 k x 64 bit external DMA I/O data memory. E. Siskind, June 12, 2003 10

SLAC FECC 3 Intelligence • IBM PPC 405 250 MHz 32 -bit Power. PC embedded in Xilinx FPGA on 3 rd generation FECC. • 16 k x 64 bit on-chip program memory; 16 k x 32 bit on-chip data memory. • 2048 k x 64 bit external program/data memory with DMA link/CAMAC hardware. E. Siskind, June 12, 2003 11

SLAC User FECC Programming • Real-time executive plus MBCD/Bitbus emulation uses ~15% of on-chip program memory, ~25% of on-chip data memory, and ~5% of off-chip RAM. • Remaining minimum of 15+ megabytes of memory on 250 MHz 32 -bit processor exceeds capacities of existing 386/486 CPU boards. • Move existing 360 Hz interrupt driven code from micro to CAMAC interface to minimize CAMAC access delay. • C run-time environment with dynamic heap allocation; full environment and stack preserved over context switch. E. Siskind, June 12, 2003 12

SLAC nd 2 Generation Hardware • PCIL 2 uses 64 bit/66 MHz PCI slot. • Single specialized fiber optic link for PCIL 2 FECC 3 communication. • 125 megabyte/second full duplex link, but 9/19 used for hardware overhead. • All peripherals FPGA-based DMA except for Bitbus (programmed I/O). • Hardware based on one large FPGA per board, plus SHARC on PCIL 2. E. Siskind, June 12, 2003 13

SLAC nd 2 Generation Status • PCIL 2 now debugged and tested. • Two PCIL 2 boards built in 2001; three more in 2002. • FECC 3 design in completed in 2002; FPGA delivery currently 29 August (5 month delay). • Initial build of two FECC 3 boards in 2003; three more after debugging. E. Siskind, June 12, 2003 14

SLAC Bitbus Master • Existing master based on MCS-51 2 MHz 8 bit microcontroller with SDLC/HDLC serial link. • 8044 BEM is an MCS-51 with mask-programmed firmware (i. DCX-51 RTE plus Bitbus application). • FECC 3 has dual-channel programmed I/O hardware (one Power. PC FIFO read or write per 32 bits) to send a packet to one slave and receive that slave’s response. • Poll list, counters, etc. maintained by new Power. PC code in FECC 3. • Enhanced real-time support via reserved high priority buffers and multiple prioritized queues. E. Siskind, June 12, 2003 15

SLAC Schedule • Migration of 360 Hz processing in progress (TEG). • All optimizations not necessary at early stage. • Trying to deploy MPG plus one ordinary PC -micro in development system in January, 2004. • Hoping to replace MPG plus one ordinary micro in summer of 2004. E. Siskind, June 12, 2003 16

SLAC Simplified Hardware Configuration • No PC PC fast feedback (KISnet equivalent) network traffic. • Use 2 nd generation PCIL-FECC hardware: – Include IEEE CAMAC and Bitbus; – Need PC with 64 bit/66 MHz PCI slot. • MPG PC (PNET equivalent) pattern broadcast on dedicated point-to-point Ethernet link. • PC-based MPG (MP 10/MP 11) broadcasts pattern to existing micros via FECC 3 BITbus interface reconfigured as a PNET transmitter; talks to SP 00/SP 01 and BIC via PC VME reflective memory links. E. Siskind, June 12, 2003 17

SLAC Integration Projects • Build, boot, and debug i. RMX-III in off-theshelf personal computers. Provide minimal FECC debugging support. • Make real-time micro data communications streams coexist with slower back-end front-end traffic on switched TCP/IP network (deferred). • Move 360 Hz interrupt processing from micro to FECC (MPG + pattern consumer). E. Siskind, June 12, 2003 18

SLAC Integration Projects II • Change control of 360 Hz interrupt processing from shared memory to network link model. • Integrate PCIL-FECC CAMAC/Bitbus emulation. • Add support for real-time CAMAC/Bitbus; Modify fast feedback actuator to specify real-time priority for CAMAC/Bitbus operations. • Modify slower micro jobs to send larger packages to CAMAC interface to maximize parallelism. • Modify klystron job to protect PIOP CAMAC operations from interrupts. E. Siskind, June 12, 2003 19

SLAC Integration Projects III • Support logical to physical micro mapping. • Support PC access to file server delivering current versions of PCIL and FECC executable images appropriate to different types of micros. E. Siskind, June 12, 2003 20