An Efficient Shared Memory Based Virtual Communication System
An Efficient Shared Memory Based Virtual Communication System for Embedded SMP Cluster Wenxuan Yin Institute of Computing Technology Chinese Academy of Sciences Joint work with Xiang Gao, Xiaojing Zhu, ICT, CAS and Deyuan Guo, Tsinghua University NAS 2011
Background • Dilemma in Embedded System – High performance – Cost, power consumption, size, etc. Video/media processing July 2011 Space-born satellite Wenxuan Yin-NAS 2011
Background • Why SMP cluster is popular in general computing? – High scalability – Good cost-performance ratio – Convenient for MPI programming • It can also benefit the embedded domain – Embedded Cluster • Embedded processor nodes • Commodity networks Tradeoff – moderate performance July 2011 cost/power efficiency Wenxuan Yin-NAS 2011
Motivations • Challenges by SMP nodes – Two levels of communication • inter-node: high-speed network • intra-node: shared memory/cache Performance Gap! – Memory management • memory hierarchy: local vs. remote • coherency maintenance – MPI Inter-Process Communication (IPC) • process allocation in different parallelism – Mutual exclusion and synchronization July 2011 Wenxuan Yin-NAS 2011
Motivations • Opportunities in SMP nodes – More computation capacity – High-speed chip-to-chip interconnect fabrics • PCI-E：ARM Cortex A 9 MPCore • Serial Rapid. IO：Freescale 8641 D • Hyper. Transport：ICT Godson-3 A – Can we use the fabrics directly to replace traditional NIC based networks? • get rid of NICs, switches, cables How to do? July 2011 Wenxuan Yin-NAS 2011
Proposed Design Extending the Shared Memory Mechanism into Inter-Node Communications July 2011 Wenxuan Yin-NAS 2011
Objectives • Compatibility – Software virtulized network TCP/IP protocol • Efficiency – Remote memory Logical shared memory – Narrow the gap between two levels • Economization – Compact interconnect Space and cost effective July 2011 Wenxuan Yin-NAS 2011
Comparison • Chip-to-chip interconnection changes the network topology Star UN Mesh … UN Ethernet Switch UN UN … UN = Uniprocessor Node July 2011 G HT G Virtual Ethernet HT HT G G = Godson-3 A SMP Wenxuan Yin-NAS 2011
Architecture Godson-3 A SMP Nodes Shared Memory Virtual Network HT 0: for interconnection Configured into 2 parts HT 1: for IO extention Omitted here Memory in each node is divided into 2 parts July 2011 Wenxuan Yin-NAS 2011
SMP Nodes • Godson-3 A CPU – MIPS 64 -compatible – 4 -core superscalar – For high performance and low power consumption Godson-3 A July 2011 Wenxuan Yin-NAS 2011
More Details • Cache coherency – Directory based cache coherency – HT holds coherency in the whole interconnection system, global addressing in remote accessing – Transparent to programmers • Reconfigurable memory pool – Each node can tune its shared memory size contributing to the memory pool – Extreme case: only master node cedes its shared part July 2011 Wenxuan Yin-NAS 2011
X-Y Transmission • Built-in routing mechanism in HT – Eliminate switches Examples G 0 → G 3 G 0 HT G 2 July 2011 HT Virtual Ethernet HT G 1 HT G 3 Wenxuan Yin-NAS 2011 G 3 → G 0
SMVN Driver • Hierarchical design – Virtual physical layer • Memory copy & optimization – Virtual data link layer • Function and hardware abstraction • Packets encapsulation meet frame format of TCP/IP – Driver management layer • Treat SMVN as a common NIC class device • OS inquiry them recurrently to load & start – Splice SMVN and TCP/IP together! July 2011 Wenxuan Yin-NAS 2011
SMVN Driver TCP/IP upper protocol SMVN July 2011 Wenxuan Yin-NAS 2011
Communication • How to implement the communication across networks? Ethernet or others July 2011 Wenxuan Yin-NAS 2011
Memory management • Data structures on SMVN buffer – Singly Linked List (SLL) Shared memory pool →L Packet …… Packet ØFree. List: global, unique ØInput. List: each node maintains one head Packet July 2011 tail Packet …… Packet No Extra Memory Allocation! Packet Wenxuan Yin-NAS 2011
Packets transmission Examples Node 0 as a sender Node 1 as a receiver 1. Free. List holds all data, Input. List is NULL 2. Sending: fetch (Free. List), copy, insert (Input. List), trigger an interrupt 3. Receiving: fetch (Input. List), copy, insert (Free. List) July 2011 Wenxuan Yin-NAS 2011
Optimization • Essentially an optimization to memory operations! • Increase the concurrency – Pipelining effect • Minimize memory access numbers – Zero-copy scheme • Reduce memory access time – Instruction-level optimization July 2011 Wenxuan Yin-NAS 2011
Concurrency • Overlap SEND/RECV operations! Pipelining effect! concurrency serial July 2011 Wenxuan Yin-NAS 2011
Zero-Copy • Change the head/tail pointers • Change the relationship which list the packets belong to tail head Free. List Input. List Packets migration Shared memory pool (L) Data copy SMVN mem pool Only scenario Network mem pool • Extra benefit: reduce power consumption! July 2011 Wenxuan Yin-NAS 2011
Bottom Optimization • To accelerate memcpy – Using cache coherency maintained by hardware • Using cached address space • Do not need flush/invalidate by programmers – Godson-3 A double-word (64 bit) RW – Unaligned memory access July 2011 Wenxuan Yin-NAS 2011
Mutual Exclusion • Why we need this? – Concurrency leads to an unpredictable outcome • Solution: spinlock – Keep atomic in shared resources operations – Test-And-Set (TAS) primitive – In Godson-3 A nodes • ll (load-linked) & sc (store-conditional) instruction pair July 2011 Wenxuan Yin-NAS 2011
Simple Lock TAS primitive • ll will record address while loading • sc can judge whether the address is modified by competitive accesses • If NO, store successively • If YES, mark a failure status in a register implicitly July 2011 Wenxuan Yin-NAS 2011
Synchronization • Occur between nodes in SMVN initialization – Master node initializes the shared memory pool, others must wait until the pool is available • When master is ready G 0 HT G 2 July 2011 HT Virtual Ethernet HT G 1 Broadcast ready status HT Activate a timer G 3 • SMVN need restart if timeout Wenxuan Yin-NAS 2011
MPI Processes • Worker Process (WP) – Its number decides the parallel degree – Real working process • Daemon Process (DP) – Its mapping decides WP’s allocation which reflects the parallel granularity • Intra-node or inter-node – At most one DP starting in each node – At least one DP residing in the cluster July 2011 Wenxuan Yin-NAS 2011
Mapping & Allocation • Mapping DPs into a binary tree connection • WP is allocated to nodes with DPs in breadth-first traversal algorithm More than 1 July 2011 DP, 1 ≤ m ≤ 4 WP, n ≥ 1 Node(i): num of WPs on Node I 0≤i≤ 3 OS SMP scheduling! Wenxuan Yin-NAS 2011
Real Platform Port MPICH 2 library in our real system Based on socket interface supported by SMVN Shared Memory Virtual Network July 2011 Godson-3 A SMP Node Wenxuan Yin-NAS 2011
Performance tests • Benchmark – OMB micro-benchmarks for MPI IPC evaluation – We choose two metrics • Ping-pong latency • Unidirection bandwidth • Performance comparison between – Inter-node vs. intra-node – Cached vs. uncached July 2011 Wenxuan Yin-NAS 2011
Testbed Setup • Towards the embedded environment – Frequency: 525 MHz – Cache size • L 1: 64 KB× 2 (including instruction and data) • L 2: 4 MB – Memory size • local in real-time OS kernel is 256 MB • shared for SMVN buffer is 2 MB – DDR 2 working at 200 MHz – HT frequency: 800 MHz July 2011 Wenxuan Yin-NAS 2011
Results-Latency cliffy smooth Basic latency July 2011 Wenxuan Yin-NAS 2011
Results-Bandwidth 32. 5 MB/s 84% 27. 3 MB/s July 2011 Wenxuan Yin-NAS 2011
Observations twice • Much better than Fast Ethernet (100 Mb) typically More than used in traditional embedded clusters – Cache is helpful! Avoid flush/invalidate by software – Tradeoff between performance and embedded constraints • Narrow the gap between two levels 84% approximability – Even superior than some high-end system although our absolute performance is lower – Introduce shared memory in both intra- and inter-node communications – Compact mesh topology in system July 2011 Wenxuan Yin-NAS 2011
Related Works • Comparison of data transfer methods – User/kernel level shared memory [Buntinas et al. ] – High-speed NIC based copy • MPI communication system (shared memory) – Nemesis [Buntinas et al. ] – High-performance and good scalability system [Chai et al. ] • RDMA system – Infini. Band [Mamidala et al. ] – Quadrics Qs. Net. II[Qian et al. ] July 2011 Wenxuan Yin-NAS 2011
Conclusion • Proposed a novel shared memory based virtual communication system --- SMVN • Goal: make a uniform infrastructure in different communication levels to implement efficient MPI IPC under embedded constraints – Adequate performace – Compact size, low power consumption, low cost (no NICs, no switches, no cables) • Direction: scalability for large system expansion July 2011 Wenxuan Yin-NAS 2011
Thanks for your attention! Questions?
- Slides: 35