1 67 Tizen v 2 3 Linux Kernel

1 67 Tizen v 2. 3 Linux Kernel Sungkyunkwan University Embedded Software Lab. @ SKKU

Tizen v 2. 3 Kernel Additional features 2 • Memory Management • CMA (Contiguous Memory Allocator), IOMMU, dma_buf • ARM Linux has no DMA_ZONE on buddy • Some devices require physically contiguous memory (CMA), other devices has IOMMU thus be able to use non-contiguous memory • Sharing DMA memory buffer for D/D to user • Graphics • DRM (Direct Rendering Manager) • Power management • CPUfreq, Devfreq, PM-Qo. S, Charger-manager • KDBUS • Multimedia • V 4 L 2 (Video for Linux 2) • EXTCON Embedded Software Lab. @ SKKU 67

Previous Reference Kernel 3 • Linux 3. 0. 15 – Obsolete LTS (Current: 3. 4 & 3. 10) • Support RD-PQ (Tizen 2) & RD-210 (Tizen 1 & 2) – RD-PQ: Exynos 4412 – RD-210: Exynos 4210 (Linux 2. 6. 36 for Tizen 1) • Not Good as Reference – – Too many backported features. Too OLD! No LTS/LTSI support Many kernel hacks & dirty patches git history removed. • Forked from production kernel. • Hard to read Embedded Software Lab. @ SKKU 67

Linux Kernel. LTS? LTSI? • Long-Term Stable (LTS) – – Maintained by Greg K. H. Bugfixes for 2 years or longer. Up to 2 LTS kernels at the same time. Recent: 3. 10. 39 (2014/5/6) • Long-Term Stable Initiative (LTSI) – Maintained by Greg K. H. and some manufacturers – Forked LTS for Industry. (LTS + Industry Patchset) – Maintain a common Linux base for use in a variety of consumer electronics products – Longer support period. – Recent: 3. 10. 31 -LTSI (2014/2/24) Embedded Software Lab. @ SKKU 4 67

Status of Tizen Reference Kernels 5 • Two reference kernels: ARM/Intel • ARM (armv 7, aarch 64) – Linux 3. 10. y • 3. 10. 33 @ 2014/05 • Full git history – Armv 6 support (Raspberry Pi) Coming soon. – Test & validation phase (integration test with userspace) • Intel (x 86, x 86_64) – Linux 3. 14. 1 – Recent ATOM Soc support merged Embedded Software Lab. @ SKKU 67

Tizen v 2. 3 Kernel Overview Embedded Software Lab. @ SKKU 6 67

Tizen v 2. 3 Kernel Overview (Cont’d) Embedded Software Lab. @ SKKU 7 67

Static Memory vs. CMA memory Multimedia example dma_alloc_coherent() Embedded Software Lab. @ SKKU 8 67

ION 9 67 • generalized memory manager that Google introduced in the Android 4. 0 ICS • Provides way for userland to allocate buffers from various “pools of memory” (aka: heaps) • CMA also allows for large physically contiguous memory allocations by migrating memory to make room for the large allocation – CMA is kernel-internal only for now, and doesn't have a interface to allow userland to allocate buffers or specify constraint options – Migrating pages to make room can cause non-deterministic delays. Embedded Software Lab. @ SKKU

ION 10 67 Embedded Software Lab. @ SKKU

11 67 Tizen v 2. 3 Memory Management Embedded Software Lab. @ SKKU

Tizen v 2. 3 Memory Management • Coupled with Graphics & Multimedia devices – Graphics & Multimedia devices = DMA devices with HUGE Buffers Embedded Software Lab. @ SKKU 12 67

What is DRM 13 67 • DRM is not Digital Right Management • Direct Rendering Manager – Kernel-level graphics device driver that support Direct Rendering Infrastructure. – Direct Rendering • An application communicates directly with graphics device driver – Display mode setting – Graphics memory management Embedded Software Lab. @ SKKU

Why DRM 14 67 • Can give display experiences similar to PC environment. • Why not use DRM until now? – DRM was designed for PC – Embedded System • Low performance • No dedicated graphics memory • Not one graphics card but separated hardware devices – Linux frame buffer driver Embedded Software Lab. @ SKKU

Why DRM (Cont’d) • Changed embedded environment – Powerful embedded So. Cs – Requirements • • Display hotplug & clone, exteded mode Unified memory management Direct rendering Varying device control with common interface Embedded Software Lab. @ SKKU 15 67

X 11 infrastructure 16 67 DIX (Device-Independent X) DDX (Device-Dependent X), Embedded Software Lab. @ SKKU

Early implementation of the Linux graphics stack 17 67 UMS (User-space Mode-Setting) Embedded Software Lab. @ SKKU

Early implementation of the Linux graphics stack 18 67 • XFree 86 server: the first Linux 2 D graphics hardware acceleration – Super-user privileges – Access the card directly from user space, no kernel support – Simple and portable • Utah-GLX: the first hardware-independent 3 D accelerated design – User space 3 D driver, directly accesses the graphics hardware from user space – 3 D hardware was clearly separated from 2 D (3 Dfx), completely separate driver. • Framebuffer drivers – Another component that could simultaneously access the graphics hardware directly. – To avoid potential conflicts between the framebuffer and XFree 86 drivers, – On VT(virtual terminal) switches, the kernel would emit a signal to the X server telling it to save the graphics hardware state. – More fragile and bug-prone drivers. • Drawbacks – Security: unprivileged user space applications be allowed to access the graphics hardware for 3 D. – Performance: all GL acceleration had to be indirect through the X protocol Embedded Software Lab. @ SKKU

DRI model 19 67 Embedded Software Lab. @ SKKU

DRI/DRM infrastructure 20 67 • • • To address the reliability and security concerns with the Utah-GLX model Additional kernel component whose duty is to check the correctness of the 3 D command stream, security-wise. Instead of accessing the card directly, the unprivileged Open. GL application would submit command buffers to the kernel, which would check them for security and then pass them to the hardware for execution. Trusting user space is no longer required. Kernel Mode-Setting (KMS) – Merge the kernel framebuffer functionality into the DRM module – X. Org access the graphics card through the DRM module and run unprivileged. – DRM module is now responsible for providing mode-setting services both as a framebuffer driver and to X. Org. Mode-setting: configuring video mode to get a picture on the screen. includes choosing the video resolution and refresh rates. Embedded Software Lab. @ SKKU

Framebuffer Drivers 21 67 • Kernel graphics driver exposing its interface through /dev/fb* • Despite their simplicity, framebuffer drivers are still a relevant option for basic 2 D display. • Especially for embedded systems – memory footprint is essential – do not require advanced graphics acceleration. • Functionalities – Mode-setting – Optional 2 D acceleration • http: //www. linux-fbdev. org/HOWTO/index. html Embedded Software Lab. @ SKKU

Direct Rendering Manager 22 67 • • • Put critical initialization of the card in the kernel, for example uploading firmwares or setting up DMA areas. Share the rendering hardware between multiple user space components, and arbitrate access. Enforce security by preventing applications from performing DMA to arbitrary memory regions, and more generally from programming the card in any way that could result in a security hole. Manage the memory of the card, by providing video memory allocation functionality to user space. More recently, the DRM was improved to achieve mode-setting. – Simplifies the previous situation where both the DRM and the framebuffer driver were fighting to access the same GPU. – Instead, the framebuffer driver is removed and framebuffer support is implemented in the DRM. Libdrm interfaces between user space and the DRM module, unprivileged user space component Embedded Software Lab. @ SKKU

DRM of Tizen v 2. 3 kernel 23 • Currently, DRM support is only for Exynos So. Cs. (ARM based) – Need Exynos specific DRM driver • Exynos DRM driver – – To support graphics hardware of Exynos So. Cs First ARM So. C graphics driver to use the DRM Merged into the mainline linux 3. 2 kernel linux/drivers/gpu/drm/exynos Embedded Software Lab. @ SKKU 67

Graphics (DRM / GEM) 24 67 • Graphics: – DRM (Direct Rendering Manager) / GEM (Graphics Execution Manager) GEM • Framework developed by Intel • To manage graphics memory • Framework for buffer management • Allocation and sharing Embedded Software Lab. @ SKKU

DRM / GEM Allocation 25 GEM Allocation steps @ Tizen (Generic) 1. DRM_IOCTL_MODE_CREATE_DUMB – Create GEM object(global) & user GEM handle(per process) • dumb_create() of struct drm_driver – No physical memory allocated. 2. DRM_IOCTL_MODE_MAP_DUMB – Create fake mmap offset of a gem object and relay the object to user • A hash key of the gem object. • dumb_map_offset() of struct drm_driver 3. MMAP – Request mmap based on the hash key as the offset – Create user address space – Setup cache attribute. 1. Not mapped to physical memory, yet Embedded Software Lab. @ SKKU 67

DRM / GEM Allocation (Cont’d) GEM Allocation steps @ Tizen (Generic) 4. On-demand Paging – Implement & Register a fault handler that • With a page fault, allocate a page and map the page. – vma->vm_ops->fault = xxx_drm_gem_fault 5. Use! 6. DRM_IOCTL_MODE_DESTROY_DUMB – Remove GEM handle & object – Free memory – Implement dumb_destroy() of struct drm_driver Embedded Software Lab. @ SKKU 26 67

DRM / GEM Allocation (Cont’d) GEM Allocation steps @ Tizen (Exynos Only) 1. DRM_IOCTL_EXYNOS_GEM_CREATE – Only user-desired size and buffer types. – Create gem object(global) & user gem handle(per process) – physical memory allocated. 2. DRM_IOCTL_EXYNOS_GEM_MMAP – Create user address space – Map the user address space to physical memory • LIBDRM of Exynos uses these APIs, not the generic. Embedded Software Lab. @ SKKU 27 67

DRM / GEM Sharing @ Tizen • • • DRM_IOCTL_GEM_FLINK – “I will share this GEM to others. ” – Create GEM object name for the given GEM handle • Global key value for sharing DRM_IOCTL_GEM_OPEN – “I want to use the shared GEM. ” – Create GEM handle based on the given GEM object name DRM_IOCTL_GEM_CLOSE Embedded Software Lab. @ SKKU 28 67

Tizen v 2. 3 Kernel MM –Multimedia (V 4 L 2 / VB 2) Embedded Software Lab. @ SKKU 29 67

V 4 L 2 / VB 2 30 67 • Tizen recommends to use V 4 L 2 at Tizen kernel for Multimedia devices – Video input (codec & camera) & Radio – However, as long as the kernel has. . : • Gstreamer/Open. MAX plugins • A method to share with other F/W via DMABUF of UMM • If V 4 L 2/VB 2 is used, things get easier. Embedded Software Lab. @ SKKU

Tizen v 2. 3 Kernel MM –Open. GL / G 3 D-GPU 31 67 Embedded Software Lab. @ SKKU

Open. GL / G 3 D-GPU 32 • Most ARM So. C GPUs (MALI, SGX, …) use their own memory manager – For mali 400 GPU in Exynos 4412 – Three packages (for kernel 3. 10) • libump-mali-400. rpm (Unified Memory Provider Library) • libtbm-exynos 4412. rpm (Tizen Buffer Manager Library) • gpu-ddk-mali-400. rpm (X 11 Driver Development Kit) – Source code is under the standard ARM commercial license to Mali GPU customers – Mali DDK modified to be compatible with UMM-DMABUF. • If GPU drivers use DRM, it would be great. – (and make them GPL) Embedded Software Lab. @ SKKU 67

Graphic and Display Usage Scenario Embedded Software Lab. @ SKKU 33 67

Graphic and Display Usage Scenario Embedded Software Lab. @ SKKU 34 67

Graphic and Display Usage Scenario Embedded Software Lab. @ SKKU 35 67

Graphic and Display Usage Scenario Embedded Software Lab. @ SKKU 36 67

Graphic and Display Usage Scenario Embedded Software Lab. @ SKKU 37 67

Tizen v 2. 3 kernel Buffer Sharing Embedded Software Lab. @ SKKU 38 67

Tizen v 2. 3 Kernel Buffer Sharing (Cont’d) 39 • Requirement from Tizen platform and hardware – Different Memory Managers: GEM, VB 2, GPU-adhoc, … – Share buffers • w/o memcpy • From and to users • Never expose directly to users (e. g. , physical address) UMM DMABUF! Embedded Software Lab. @ SKKU 67

Unified Memory Management (UMM) • Introduced by Jesse Barker, 2011 • Includes – DMABUF (sharing buffer) • Export – GEM/VB 2/… object DMABUF • Import – DMABUF GEM/VB 2/… object • Userspace sees DMABUF as a File Descriptor – DMA mapping API for allocation – CMA (Contiguous Memory Allocator) – IOMMU (MMU for I/O devs) Embedded Software Lab. @ SKKU 40 67

DMA Buffer Sharing: Camera Display 41 67 Embedded Software Lab. @ SKKU

DMA Buffer Sharing Example • Camera App 1) Request V 4 L 2 camera buffer (U) 2) Allocate CMA buffer (K) 3) Request a camera frame at the V 4 L 2 buffer (U) 4) Store the camera frame & Notify user (K) 5) Request DMABUF export for the V 4 L 2 camera buffer (U) 6) dma_buf_exporter() (K) • Create DMABUF from V 4 L 2 buffer 7) dma_buf_fd() (K) • Provide FD of the DMABUF to user Embedded Software Lab. @ SKKU 42 67

DMA Buffer Sharing Example (Cont’d) • Camera App 1) Request V 4 L 2 camera buffer (U) 2) Allocate CMA buffer (K) 3) Request a camera frame at the V 4 L 2 buffer (U) 4) Store the camera frame & Notify user (K) 5) Request DMABUF export for the V 4 L 2 camera buffer (U) 6) dma_buf_exporter() (K) • Create DMABUF from V 4 L 2 buffer 7) dma_buf_fd() (K) • Provide FD of the DMABUF to user Embedded Software Lab. @ SKKU 43 67

DMA Buffer Sharing Example (Cont’d) • Camera App 1) Request V 4 L 2 camera buffer (U) 2) Allocate CMA buffer (K) 3) Request a camera frame at the V 4 L 2 buffer (U) 4) Store the camera frame & Notify user (K) 5) Request DMABUF export for the V 4 L 2 camera buffer (U) 6) dma_buf_exporter() (K) • Create DMABUF from V 4 L 2 buffer 7) dma_buf_fd() (K) • Provide FD of the DMABUF to user Embedded Software Lab. @ SKKU 44 67

DMA Buffer Sharing Example (Cont’d) • Camera App 8) Request FD->GEM conversion (U) 9) dma_buf_get(fd) (K) • Get DMABUF from FD 10) dma_buf_attach(dma-buf) / dma_buf_map_attachment() (K) • Get Buffer from DMABUF 11) Import as GEM, send user (K) 12) Request GEM object name (U) 13) Return GEM object name (K) 14) Send GEM object name to X (U) Embedded Software Lab. @ SKKU 45 67

DMA Buffer Sharing Example (Cont’d) • Camera App 8) Request FD->GEM conversion (U) 9) dma_buf_get(fd) (K) • Get DMABUF from FD 10) dma_buf_attach(dma-buf) / dma_buf_map_attachment() (K) • Get Buffer from DMABUF 11) Import as GEM, send user (K) 12) Request GEM object name (U) 13) Return GEM object name (K) 14) Send GEM object name to X (U) Embedded Software Lab. @ SKKU 46 67

DMA Buffer Sharing Example (Cont’d) • X server 15) Convert given GEM object name to GEM. Display its content. (U & K) Embedded Software Lab. @ SKKU 47 67

DMA Buffer Sharing Example (Cont’d) • Close after usage 1) “Free” the GEM object (U) 2) Remove reference from the DMABUF (K) 3) Close(DMABUF-FD) at cam app (U) 4) (No more reference to DMABUF) Release callback executed (K) Embedded Software Lab. @ SKKU 48 67

Buffer Synchronization Embedded Software Lab. @ SKKU 49 67

Problem Definition 50 67 • Simple Usage Scenario – 1. CPU writes to buffer 1. – 2. CPU tells GPU to do something on buffer 1. – 3. GPU does something on buffer 1. • GPU finishes. – 4. CPU reads the buffer 1 How to ensure CPU won’t use buffer 1 until step 4? CPU GPU 1 4 2 BUS DRAM Embedded Software Lab. @ SKKU 3

Problem Definition (Cont’d) 51 • What if there are two threads using GPU? – And if the two look at the same buffer? • Not even aware of it? • What if the two DMA devices (GPU, FIMC) share buffer? – – But hard to know it at drivers or user threads. FIMC never knows when GPU finishes FIMC never knows when Threads 1 stops using “buffer 1” Threads 2 never know when GPU stops using “buffer 1” Embedded Software Lab. @ SKKU 67

Buffer Synchronization • TGL (Tizen Global Lock) @ Tizen 2. 0 – … Let userspace handle the issue …: dma_buf_sync – Kernel patch required. • Sync Framework (Google) – Jun, 2012. Resources w/o DMABUF (similar with TGL) • KDS (Kernel Dependency System, ARM) – May 2012 / DMABUF-compatible • DMA Fence Framework (Canonical/TI/Samsung) – Aug 2012 / DMABUF-compatible – Jun 2013 / DMABUF-Sync Framework (Samsung) – https: //bugs. tizen. org/jira/browse/PTF-11 Embedded Software Lab. @ SKKU 52 67

53 67 K-dbus Embedded Software Lab. @ SKKU

Kdbus –D-bus is? • D-Bus – – Message bus system Method Call transaction Signals Broadcasting Embedded Software Lab. @ SKKU 54 67

Kdbus –K-dbus is? • Low-level, native kernel D-Bus transport – All communication between processes take place over special character device nodes in /dev/kdbus • kernel copies the message from user space directly into the receive buffer for the destination process. • Single copy to destinations • memfds – File descriptor for memory regions – Zero copy implementation – Similar to ashmem in Android http: //lwn. net/Articles/580194/ http: //en. wikipedia. org/wiki/User: Scot. XW/kdbus Embedded Software Lab. @ SKKU 55 67

D-bus vs Kdbus Embedded Software Lab. @ SKKU 56 67

57 67 Others Embedded Software Lab. @ SKKU

EXTCON (External Connector) 58 • Extension of Android kernel's switch class located at linux/drivers/switch/ • Manage status of cable & ports – A port with multiple cables (docks, multi-cables, …) – A port with multiple modes (USB, HDMI, TA, …) • 3. 5 pi: stereo, stereo+mic+buttons, stereo+buttons, mono, … • Compatible with Switch – Android Switch drivers can be easily ported • Refer to Linux/Documentation (porting guide for switch driver) – Extcon drivers export both Switch and Extcon interfaces (compat mode) • In Reference Device – MUIC (USB+HDMI+TA+DOCK+…) – 3. 5 Pi Jack Embedded Software Lab. @ SKKU 67

Charger 59 67 • Charger Manager • /drivers/power/charger-manager. c – All needed by Tizen user space are prepared – No OAL modification required • OR supply battery/charger interface with power-supply-class • Use EXTCON for Cable-Input (MUIC in mobile) – Switch class is no longer available in Linux. • Note: some SOC (state-of-charge) value is required for mobile profile. Unless, Tizen will assume that SOC is 0 Shutdown! Embedded Software Lab. @ SKKU

Power Management (Tizen 2. x) • Tizen v 2. 3 Kernel does not use opportunistic sleep like Android wake-lock • Userspace(Power Manager) should request sleep directly Embedded Software Lab. @ SKKU 60 67

Power Management (Tizen 3. 0) 61 67 • Components (e. g. , OOM) are merged and consolidated into Resourced. • Tizen v 3. 0 use deviced ØDevice control access is merged into deviced • Tizen v 2. x use Power Manager ØApp can access kernel device node by using devman library Embedded Software Lab. @ SKKU

Power Management 62 • Recommendation For Tizen 3. 0 or later – Do not use DVFS (CPUfreq/Devfreq) min/max ABIs • PASS (Power-Aware System Service in Deviced) • PASS is a module in deviced/resourced (TBD, currently prototyped in system-server of Tizen 2. 2) that monitors the userspace activities and controls #cores & DVFS conditions. – Use (keep their standard ABIs for PASS) • CPUfreq& Devfreq (DVFS for non CPU devices if you have them) • Thermal FW – PASS gives hints to DVFS/Hotplug • based on the info from user space. • based on the other kernel ABIs (e. g. , Thermal) • highly configurable. (control knob of performance & power) Embedded Software Lab. @ SKKU 67

Other Peripherals: Sound 63 • Use ALSA Scenario (www. alsa-project. org) 67 – Make hardware plugin and device driver based on ALSA standard. • https: //wiki. tizen. org/wiki/Porting_Guide#Audio Embedded Software Lab. @ SKKU

Other Peripherals: Blue. Tooth 64 • Use Bluez(www. bluez. org) • https: //wiki. tizen. org/wiki/Porting_Guide#Bluetooth Embedded Software Lab. @ SKKU 67

Other Peripherals: Wi. Fi • Use wpa-supplicant • https: //wiki. tizen. org/wiki/Porting_Guide#WLAN Embedded Software Lab. @ SKKU 65 67

Core Daemons (Tizen 3. 0) • Systemd (booting & daemon management) • Deviced/Resourced – Power-manager, devman, and system-server are consolidated. • DBUS: The usage of Vconf is reduced Embedded Software Lab. @ SKKU 66 67

OEM Adaptation Layer (OAL) • • 67 https: //wiki. tizen. org/wiki/Porting_Guide#System_OAL Tizen 2. 0/adaptation/* – Related to System FW: • Tizen 2. 0/adaptation/ap_samsung/device-manager-plugin-exynos • Tizen 2. 0/adaptation/intel_mfld/device-manager-plugin-mfldblackbay – Related to Alsa (sound) : • Tizen 2. 0/adaptation/devices/alsa-scenario-* Embedded Software Lab. @ SKKU 67