An FPGA Implementation of the Smooth Particle Mesh

An FPGA Implementation of the Smooth Particle Mesh Ewald Reciprocal Sum Compute Engine (RSCE) Sam Lee

What is this Thesis about? p Implementation n n p Reciprocal Sum Compute Engine (RCSE). FPGA based. Accelerate part of Molecular Dynamics Sim. Smooth Particle Mesh Ewald. Investigation n Precision requirement. Speedup capability. Parallelization strategy. 2

Outline p What is Molecular Dynamics Simulation? p What calculations are involved? p How do we accelerate and parallelize the calculations? p What did we find out about precision? p What did we find out about speedup? p What is left to be done? 3

Molecular Dynamics Simulation 4

Molecular Dynamics Simulation p p p E = - V (Electric Field = -Gradient of Potential) F = QE (Force = Charge x Electric Field) F = ma (Force = Mass x Acceleration) Time integration => New Positions and Velocities ∆ p A Force by A on C 1+ 2 - Force by C on A B A on e by Forc A B on e by Forc e rc Fo by C e rc Fo on by C B B on C 1+ B 5

MD Simulation p Problem scientists are facing: n SLOW! n O(N^2). n N=105, time-span=1 ns, timestep size=1 fs => 1022 calculations. n An 3 GHz computer takes 5. 8 x 1012 days to finish!! 6

Solution p Accelerate with FPGA p Especially: n p The O(N 2) calculations. To be more specific, thesis addresses: n n Reciprocal Electrostatic energy and force calculations. Smooth Particle Mesh Ewald algorithm. 7

Previous Work p Software Implementations: n n n p Original PME Package written by Toukmaji. NAMD 2. AMBER. Hardware Implementations: n n n No previous hardware implementations of SPME. MD-Grape & MD-Engine used Ewald Summation is O(N 2); SPME is O(NLog. N)! 8

Calculations Involved p Smooth Particle Mesh Ewald 9

Electrostatic Interaction p Coulombic equation: p Under the Periodic Boundary Condition, summation is only … Conditionally Convergent. 10

Periodic Boundary Condition p To combat Surface Effect… A 1 B 2 1 4 5 D 1 2 Replication 4 3 5 5 E 2 1 5 1 F 2 1 5 5 3 I H 2 2 1 1 4 5 3 2 4 3 5 3 4 3 2 4 3 4 G 1 4 3 1 C 2 2 4 5 3 5 11

Ewald Summation Used For PBC p p To calculate for the Coulombic Interactions. O(N 2) Direct Sum + O(N 2) Reciprocal Sum. q r Direct Sum Reciprocal Sum q q r r 12

Smooth Particle Mesh Ewald p Shift the workload to the Reciprocal Sum. p Use Fast Fourier Transform. p O(N) Real + O(NLog. N) Reciprocal. p RSCE calculates the Reciprocal Sum using the SPME algorithm. 13

SPME Reciprocal Energy FFT 14

SPME Reciprocal Force 15

Reciprocal Sum Compute Engine (RSCE) 16

RSCE Validation Environment 17

RSCE Architecture 18

RSCE Verification Testbench 19

RSCE System. C Model 20

MD Simulations with the RSCE 21

RSCE Precision Goal p Goal: Relative error < 10 -5. p Two major calculation steps: n n p B-Spline Calculation. 3 D-FFT Calculation. Due to limited logic resource + limited precision FFT Logi. Core. => Precision goal CANNOT be achieved. 22

MD Simulation with RSCE p RMS Energy Error Fluctuation: 23

FFT Precision Vs. Energy Fluctuation 24

Speedup Analysis p RSCE vs. Software Implementation 25

RSCE Speedup p RSCE @ 100 MHz vs. P 4 Intel @ 2. 4 GHz. n p Speedup: 3 x to 14 x RSCE Computation time: 26

RSCE Speedup p Why so insignificant? n n p Solution: n p QMM bandwidth limitation. Sequential nature of the SPME algorithm. Use more QMM memories. Slight design modifications required. 27

Multi-QMM RSCE Speedup p NQ-QMM RSCE Computation time : The 4 -QMM RSCE n Speedup: 14 x to 20 x. Assume N is of the same order as Kx. K: n Speedup: 3(NQ-1)x 28

RSCE Speedup N P K Single-QMM Speedup against Software Four-QMM Speedup against Single-QMM Four-QMM Speedup against Software Speedup 20000 4 32 5. 44 x 3. 37 18 x Speedup 20000 4 64 6. 97 x 2. 10 14 x Speedup 20000 4 128 10. 70 x 1. 46 15 x Speedup 20000 8 32 3. 72 x 3. 90 14 x Speedup 20000 8 64 5. 17 x 3. 37 17 x Speedup 20000 8 128 7. 94 x 2. 10 16 x x 29

Parallelization Strategy p When Multiple RSCEs are Used Together 30

RSCE Parallelization Strategy p p p Assume a 2 -D Simulation. Assume P=2, K=8, N=6. Assume Num. P = 4. An 8 x 8 x 8 mesh Four 4 x 4 x 4 Mini Meshes 31

RSCE Parallelization Strategy p p Mini-mesh composed -> 2 D-IFFT = two passes of 1 D-FFT (X and Y). X Direction FFT Y Direction FFT 32

Parallelization Strategy p p 2 D-IFFT -> Energy Calculation -> 2 D-FFT -> Force Calculation Energy Calculation Force Calculation 2 D-FFT 33

Multi-RSCE System 34

Conclusion p Successful integration of the RSCE into NAMD 2. p Single-QMM RSCE Speedup = 3 x to 14 x. p NQ-QMM RSCE Speedup = 14 x to 20 x. p When N≈Kx. K, NQ-QMM Speedup = (NQ-1)3 x. p Multi-RSCE system is still a better alternative than the Multi-FPGA Ewald Summation system. 35

Future Work p Input Precision Analysis. p More in-depth FFT Precision Analysis. p Implementation of block-floating Point FFT. p More investigation on how different simulation setting (K, P, and N) affects the RSCE speedup. p Investigate how to better parallelize the SPME algorithm. 36

Questions? 37