GPU acceleration in Matlab Jan Kamenick UTIA Friday

GPU acceleration in Matlab Jan Kamenický UTIA Friday seminar 9. 11. 2012

GPU acceleration • CPU – fast – general-purpose • GPU – highly parallel – handles specific tasks with large amount of data – memory transfers needed

GPU acceleration in Matlab • Build-in functions – many Matlab functions support GPU acceleration natively • arrayfun – specific element-wise processing • CUDA kernels – write “. cu” files – compile to “. ptx” (parallel thread execution) – run using feval

Prerequisites • Matlab 2010 b or newer • Parallel Computing Toolbox ver

Prerequisites >> ver ------------------------------------------MATLAB Version 7. 13. 0. 564 (R 2011 b) MATLAB License Number: XXXXXX Operating System: Microsoft Windows 7 Version 6. 1 (Build 7601: Service Pack 1) Java VM Version: Java 1. 6. 0_17 -b 04 with Sun Microsystems Inc. Java Hot. Spot(TM) 64 -Bit Server VM mixed mode ------------------------------------------MATLAB Version 7. 13 (R 2011 b) Simulink Version 7. 8 (R 2011 b) Computer Vision System Toolbox Version 4. 1 (R 2011 b) Curve Fitting Toolbox Version 3. 2 (R 2011 b) DSP System Toolbox Version 8. 1 (R 2011 b) Data Acquisition Toolbox Version 3. 0 (R 2011 b) Filter Design HDL Coder Version 2. 9 (R 2011 b) Fixed-Point Toolbox Version 3. 4 (R 2011 b) Global Optimization Toolbox Version 3. 2 (R 2011 b) Image Acquisition Toolbox Version 4. 2 (R 2011 b) Image Processing Toolbox Version 7. 3 (R 2011 b) MATLAB Compiler Version 4. 16 (R 2011 b) MATLAB Distributed Computing Server Version 5. 2 (R 2011 b) Neural Network Toolbox Version 7. 0. 2 (R 2011 b) Optimization Toolbox Version 6. 1 (R 2011 b) Parallel Computing Toolbox Version 5. 2 (R 2011 b) Partial Differential Equation Toolbox Version 1. 0. 19 (R 2011 b) Signal Processing Toolbox Version 6. 16 (R 2011 b) Simulink 3 D Animation Version 6. 0 (R 2011 b) Statistics Toolbox Version 7. 6 (R 2011 b) Symbolic Math Toolbox Version 5. 7 (R 2011 b) Wavelet Toolbox Version 4. 8 (R 2011 b)

Prerequisites • Matlab 2010 b or newer • Parallel Computing Toolbox ver • NVIDIA GPU with CUDA version 1. 3 or higher gpu. Device

Prerequisites >> gpu. Device ans = parallel. gpu. CUDADevice handle Package: parallel. gpu Properties: Name: Index: Compute. Capability: Supports. Double: Driver. Version: Max. Threads. Per. Block: Max. Shmem. Per. Block: Max. Thread. Block. Size: Max. Grid. Size: SIMDWidth: Total. Memory: Free. Memory: Multiprocessor. Count: Clock. Rate. KHz: Compute. Mode: GPUOverlaps. Transfers: Kernel. Execution. Timeout: Can. Map. Host. Memory: Device. Supported: Device. Selected: 'Ge. Force GTX 285' 1 '1. 3' 1 5 512 16384 [512 64] [65535] 32 2. 1475 e+009 1. 9656 e+009 30 1476000 'Default' 1 1 1 Methods, Events, Superclasses

Basic usage • Send data to GPU – either allocate there or transfer from workspace • Run Matlab functions – GPU acceleration is used automatically • Retrieve the output data

GPUArray class parallel. gpu. GPUArray – main data class for GPU computations – stored in the GPU memory – create directly using static methods zeros nan eye rand linspace ones true colon randi logspace inf false – copy from existing data gpu. Array(img) randn

GPUArray class • Supported data types: (u)int 8, (u)int 16, (u)int 32, (u)int 64, single, double, logical – determine the type using class. Underlying(gpu. Var) • Retrieve the data using workspace. Var = gather(gpu. Var)

GPU accelerated Matlab functions (2012 b) methods(‘parallel. gpu. GPUArray’)

GPU accelerated Matlab functions (2012 b) abs acosh acoth acsch all angle any arrayfun asech asinh atan 2 atanh betaln bitand bitcmp bitget bitor bitset bitshift bitxor blkdiag bsxfun cast cat ceil chol circshift class. Underlying colon complex cond conj conv 2 convn cosh coth cov cross csch ctranspose cumprod cumsum det diag diff display dot double eig eps eq erfcinv erfcx erfinv expm 1 fft 2 fftn fftshift filter 2 find fix fliplr flipud flipdim floor fprintf full gammaln gather ge gt horzcat hypot ifft 2 ifftn ifftshift imag ind 2 sub int 16 int 2 str int 32 int 64 int 8 inv ipermute iscolumn isempty isequaln isfinite isinf islogical ismatrix isnan isreal isrow issorted issparse isvector kron ldivide le length log 10 log 1 p log 2 logical lt lu mat 2 str max mean meshgrid minus mldivide mod mpower mrdivide mtimes ndgrid ndims ne nnz normest not num 2 str numel perms permute plot (and related) plus pow 2 power prod qr rank rdivide reallog realpow realsqrt rem repmat reshape rot 90 round sech shiftdim sign single sinh size sort sprintf sqrt squeeze std sub 2 ind subsasgn subsindex subsref sum svd tanh times trace transpose tril triu uint 16 uint 32 uint 64 uint 8 uminus uplus var vertcat

Simple example • Solve system of linear equations (Ax = b) A b x x = = gpu. Array(A); gpu. Array(b); Ab; gather(x);

Simple example • Compute convolution using FFT img msk I = M = res = gpu. Array(img); = padarray(msk, size(img)-size(msk), 0, 'post'); = gpu. Array(msk); fft 2(img); fft 2(msk, size(img, 1), size(img, 2)); = real(ifft 2(I. *M)); = gather(res);

Linear system solution benchmark Speedup of computations on GPU compared to CPU 3. 5 3 Speedup 2. 5 2 single-precision 1. 5 double-precision 1 0. 5 0 24 10 48 20 30 72 40 96 20 51 44 61 68 71 92 81 16 92 0 24 10 Matrix size (number of equations) 4 26 11 8 28 12 2 31 13

Convolution benchmark Speedup of computations on GPU compared to CPU 5 4 Speedup 3. 5 3 2. 5 2 1. 5 single-precision 1 double-precision 0. 5 0 8 12 6 25 4 38 2 51 0 64 8 76 6 89 10 24 12 80 15 36 17 92 Matrix size 20 48 25 60 30 72 84 35 96 40 20 51 44 61 68 71

Profiling • Before optimizing (trying to use GPU) locate promising parts of code like – custom code consuming the majority of time – build-in functions that support GPUArray (consuming the majority of time) – large input/output data, simple data types • Test the speed afterwards • GPU code cannot be profiled

Profiling