The Highend Virtualization Company IDC HPC User Forum

  • Slides: 13
Download presentation
The High-end Virtualization Company IDC HPC User Forum 09/08/2009 Manuel Hoffmann, Vice President, Channel

The High-end Virtualization Company IDC HPC User Forum 09/08/2009 Manuel Hoffmann, Vice President, Channel Development Manuel@Scale. MP. com +1. 408. 342. 0337 Aggregate. Scale. Simplify. Save. 11/26/20201

Scale. MP at a Glance Virtualization for high-end computing delivering higher performance and lower

Scale. MP at a Glance Virtualization for high-end computing delivering higher performance and lower Total Cost of Ownership through aggregation of multiple x 86 off-the-shelves servers into a single large virtual shared memory system • 100+ Deployments Worldwide • Founded in 2003 • Product shipping since 2006 • Sold through Tier-1 and Tier-2 OEMs Aggregate. Scale. Simplify. Save. 11/26/2020 2 Confidential and Proprietary

Virtualization ? • “a technique for hiding the physical characteristics of computing resources from

Virtualization ? • “a technique for hiding the physical characteristics of computing resources from the way in which other systems, applications or end users interact with these resources” Wikipedia Aggregation Partitioning Providing a virtual resource that is a subset of the physical resource Utilization Providing a virtual resource that is a concatenation of several physical resources Flexibility and Capability (Disk Partitioning) (RAID and Volume Manager) Flexibility Availability and Capability (VLANs) (Link Aggregation) Utilization (Server Virtualization) Aggregate. Scale. Simplify. Save. ? ? ? 11/26/2020 3 Confidential and Proprietary

Server Virtualization PARTITIONING AGGREGATION Subset of the physical resource Concatenation of physical resources Virtual

Server Virtualization PARTITIONING AGGREGATION Subset of the physical resource Concatenation of physical resources Virtual Machine App OS Hypervisor or VMM Aggregate. Scale. Simplify. Save. Hypervisor or VMM 11/26/2020 4 Confidential and Proprietary

How Does It Work? Multiple off-the-shelf x 86 servers, with processors and memory Processors

How Does It Work? Multiple off-the-shelf x 86 servers, with processors and memory Processors speed/amount or memory amount does not have to be same across all boards Infini. Band HCAs, cables and switch v. SMP Foundation™ Devices The flash-devices plug into the boards and used as bootable device. v. SMP Foundation is booted to present an aggregate coherent view to the OS HIGH-END X 86 SYSTEM, BASED ON STANDARD X 86 COMPONENTS Aggregate. Scale. Simplify. Save. 11/26/2020 5 Confidential and Proprietary

Behind The Scenes One System SEAMLESS INTEGRATION • Software interception engine creates a uniform

Behind The Scenes One System SEAMLESS INTEGRATION • Software interception engine creates a uniform execution environment • v. SMP Foundation creates the relevant BIOS environment to present the OS (and the SW stack above it) as single coherent system Coherent Memory • v. SMP Foundation maintains cache coherency between boards • Multiple concurrent memory coherency mechanisms, on a per-block basis, based on real-time memory activity access pattern • Leverage board local-memory for caching HIGHEST X 86 SMP MEMORY BANDWIDTH! Shared I/O • v. SMP exposes all available I/O resources to the OS in a unified PCI hierarchy • No need for cluster file systems Aggregate. Scale. Simplify. Save. 11/26/2020 6 Confidential and Proprietary

Server Virtualization Aggregation SMP Cost Savings & Performance High core-count / Large memory AGGREGATION

Server Virtualization Aggregation SMP Cost Savings & Performance High core-count / Large memory AGGREGATION Concatenation of physical resources Virtual Machine App OS Hypervisor or VMM Cluster Manageability New management paradigm for small clusters: 4 to 64 nodes Cloud Flexibility On-the-fly provisioning for compute grids: unlimited scaling Aggregate. Scale. Simplify. Save. 11/26/2020 7 Confidential and Proprietary

v. SMP Foundation Aggregation Platform SMP Cluster Cloud Cost Savings & Performance Manageability Flexibility

v. SMP Foundation Aggregation Platform SMP Cluster Cloud Cost Savings & Performance Manageability Flexibility Cost Savings • Up to 5 X cost savings Performance • Leveraging latest Intel processors • Best x 86 solution by SPEC CPU 2006 • #7 th best shared-memory by STREAM (memory bandwidth) Reliability • Fault detection and component isolation • Redundant backplane support Capabilities • Up to 128 cores and 4 TB RAM Manageability • Single Operating System (OS) for up to 16 nodes • OS driven job scheduling and resource management Performance • Infini. Band performance with zero management and knowhow Storage • Built-in cluster file system Installation • Unboxing to production in less than 3 hours Aggregate. Scale. Simplify. Save. Flexibility • On-the-fly aggregated VM provisioning and tear-down • Scaling memory or CPU Utilization • Resource fragmentation reduction • Support any programming model (serial, throughput, multi-threaded, large-memory) without machine boundary Integration • Network installation provides seamless integration with any grid management system 11/26/2020 8 Confidential and Proprietary

Target Environments and Applications Target Environments • Users seeking to simplify cluster complexities Typical

Target Environments and Applications Target Environments • Users seeking to simplify cluster complexities Typical end-user applications Manufacturing Life Sciences Energy CSM (Computational Gaussian VASP AMBER Schrödinger Jaguar Schrödinger Glide NAMD DOCK GAMESS GOLD mpi. BLAST GROMACS MOLPRO Open. Eye FRED Open. Eye OMEGA SCM ADF HMMER Schlumberger ECLIPSE Paradigm Geo. Depth 3 DGEO 3 DPSDM Norsar 3 D Structural Mechanics) ABAQUS/Explicit ABAQUS/Standard ANSYS Mechanical LSTC LS-DYNA ALTAIR Radioss NASTRAN • Applications that use large memory footprint (even with one processor) • Applications that need multiple processors and shared memory CFD (Computational Fluid Dynamics) FLUENT ANSYS CFX STAR-CD AVL FIRE Tgrid Other in. Trace Open. RT Weather Forecasting MM 5 WRF Aggregate. Scale. Simplify. Save. EDA Mentor Cadence Synopsys Finance Wombat KX Others The Math. Works MATLAB R Octave Wolfram MATHEMATICA ISC STAR-P 11/26/2020 9 Confidential and Proprietary

Example v. SMP Foundation Cluster WEATHER FORECASTING SERVICE PROVIDER • Challenges: – Need to

Example v. SMP Foundation Cluster WEATHER FORECASTING SERVICE PROVIDER • Challenges: – Need to run MPI as well as Open. MP codes – System needs to be deployed remotely, and hence needs to be simple to manage – Data processing flow is complex and requires transferring large amounts of data between steps • Applications: MM 5, WRF, MAWSIP, Home-grown code for data transformation • Solution: – 4 Intel Nehalem dual socket blades, total of 8 sockets (32 cores) and 192 GB RAM – Internal storage – Solution was extended to 8 blades, total of 16 sockets (64 cores) and 384 GB RAM • Benefits: – Performance: 2. 5 X better performance on same # of cores (32) – Simpler solution: Significantly reduced capital expense, allowed the customer to have a higher # of cores – Simplicity: Simple to manage by domain experts (weather forecast scientists) – Dataflow remains within the system, leveraging internal storage Aggregate. Scale. Simplify. Save. SIMPLE AND FLEXIBLE COST EFFECTIVE SOLUTION 11/26/2020 10 Confidential and Proprietary

Example: v. SMP Foundation SMP FORMULA 1 TEAM • Challenges: – Need to generate

Example: v. SMP Foundation SMP FORMULA 1 TEAM • Challenges: – Need to generate large mesh as part of pre-processing of whole-car simulation (FLUENT TGrid). Mesh requirements are ~200 GB in size – Expect to grow significantly within 12 months after initial deployment – Would like to standardize on x 86 architecture due to lower costs and open standards • Solution: – 12 Intel dual-processor Xeon systems to provide 384 GB RAM single virtual system running Linux with v. SMP Foundation • Benefits: – Better performance: Solution evaluated and found to be faster than alternative systems (x 86 and non-x 86) – Cost: Significant savings compared to alternative system – Versatility: Also being used to run FLUENT (MPI) as part of large cluster – Investment protection: Solution can grow Aggregate. Scale. Simplify. Save. SCALEUP AT SCALEOUT PRICING 11/26/2020 11 Confidential and Proprietary

Example: v. SMP Foundation Cluster FINANCIAL SERVICES • Challenges: – A single 4 -socket

Example: v. SMP Foundation Cluster FINANCIAL SERVICES • Challenges: – A single 4 -socket server did not provide enough performance required for customer business targets – Multiple 4 -socket servers required complex decomposition and introduced challenges in transferring data between processes in a short and deterministic time (low latency and small jitters) – Co-location at exchanges for a solution comprised of multiple systems is complicated • Applications: KX, WOMBAT, Home-grown code • Solution: – 16 Intel dual-processor Xeon systems to provide 0. 5 TB RAM, 32 sockets (128 cores) single virtual system running Linux with v. SMP Foundation • Benefits: – – Reduced latency and latency variance Simpler solution: Deploy and management of a single system Better utilization: Single system reduces resources fragmentation Simpler programming model: No need for specific Infini. Band programming Aggregate. Scale. Simplify. Save. SIMPLIFYING INTER-PROCESS COMMUNICATION 11/26/2020 12 Confidential and Proprietary

Example: v. SMP Foundation Cloud HOSTED HPC RESOURCE PROVIDER • Problems: – Need to

Example: v. SMP Foundation Cloud HOSTED HPC RESOURCE PROVIDER • Problems: – Need to provision systems for MPI as well as Open. MP (shared memory) codes – Large shared memory jobs currently require dedicated proprietary hardware – Low utilization on shared memory systems • Applications: A variety of commercial and customer codes • Solution: – Original: 4 systems, total of 8 sockets (32 cores) and 128 GB RAM – Solution was extended to 16 nodes • Benefits: – Utilization: Rely on standard commodity hardware – Flexibility: Using same system for both shared memory and cluster benchmarks, resulting in high utilization COST EFFECTIVE FLEXIBLE SOLUTION WITH HIGH UTILIZATION Aggregate. Scale. Simplify. Save. 11/26/2020 13 Confidential and Proprietary