Design and Implementation of Adaptive Signal Processing Systems

Design and Implementation of Adaptive Signal Processing Systems Using Markov Decision Processes Marilyn Wolf 1 With contributions from: Lin Li 2, Adrian E. Sapio 2, Jiahao Wu 2, Yanzhou Liu 2, Kyunghun Lee 2, and Shuvra S. Bhattacharyya 2, 3 1. Georgia Institute of Technology, USA 2. University of Maryland, USA 3. Tampere University of Technology, Finland

Outline • Motivation and Contribution • Background • Framework for Design and Implementation of Adaptive Signal Processing Systems • Case Study of Channelizer/receiver Application • Conclusions 2

Motivation and Contribution • Modern signal processing applications impose increasing demands of: – – Adaptivity Efficiency Reconfigurability Flexibility • Challenges at many levels of system design, implementation and optimization: – Dynamically-changing working scenarios – Stringent constraints on energy-efficiency and real -time performance – Multidimensional design space 3

Motivation and Contribution • This research aims at developing a framework for design and implementation of adaptive embedded signal processing systems that integrates – automated, MDP-based generation of optimal reconfiguration policies – dataflow-based application modeling – implementation of embedded control software that carries out the generated reconfiguration policies 4

Background Markov Decision Processes (MDPs) • Probabilistic transitions combined with inputs. – Given an input at a state, next state is chosen probabilistically. • A policy p defines the actions in each state s. – Optimal policy maximizes rewards. 5

Background Dataflow Modeling • Basic concepts of dataflow modeling: – Digital Signal Processing (DSP) system directed dataflow graph – Computational functions nodes (actors) – Communication channels between actors edges (FIFOs) – Actor Firing: Actor execution as a discrete unit of computation – Token: The encapsulation of some well-defined amount of data. – Consumption/Production Rate: Number of tokens consumed/produced from/to the input/output FIFO during one actor firing. 6

Background Light. Weight Data. Flow (LWDF) • Light. Weight Data. Flow (LWDF): a programming methodology for integration, experimentation, and optimization with dataflow modeling approaches. • Actor Mode: Determines the dataflow behavior of the actor. • Enable function: Checks actor firing condition according to its current mode. This function can be bypassed at run time if static scheduling analysis can ensure the result. • Invoke function: Executes an actor firing according to its current mode. • Lightweight Dataflow Environment (LIDE): – Provides a compact set of application programming interfaces (APIs) that is used for constructing, connecting, and executing dataflow components such as actors, edges, and graphs. – LIDE APIs have been implemented in a variety of implementation languages, including C, Verilog, and CUDA 7

Framework for Adaptive Signal Processing Systems We propose a novel framework, called Hierarchical MDP framework for Compact System-level Modeling (HMCSM). 8

Framework for Adaptive Signal Processing Systems • Hierarchical MDP Subsystem – A single Markov decision process (MDP) is transformed into a hierarchy of multiple MDPs that can be independently solved. – Such decomposition into a collection of simpler MDPs leads to more efficient design optimization. – The stochastic models of the environment and system include, for each of the environment and system, the definition of the state space and the state transition matrix (STM). – Control Actions represent the set of possible reconfiguration operations. – The Reward Function maps state-action pairs into scores that assess the utility of performing the associated control action during the given state. 9

Framework for Adaptive Signal Processing Systems • Parameterized LIDE Implementation – Dataflow graph implementation of the application developed using LIDE – Parameter updates are made by setting (at design time or run time) appropriate variables in this implementation • Configuration Control Machine (CCM) – Determines, based on the current environmental state and system state, whether or not to perform a dynamic reconfiguration operation – Determines the specific reconfiguration operation that is to be applied to the system if reconfiguration is to be performed • Policy Mapping Engine (MPE) – Translates control actions into updates to dynamic parameters in the embedded software 10

Case Study of Channelizer/receiver Application (1) 11

Case Study of Channelizer/receiver Application (2) Init Graph: modeling construct in parameterized synchronous dataflow (PSDF) for reconfiguration functionality (determination and propagation of new parameter values). Body Graph: modeling construct in PSDF for core signal processing functionality associated with an application. PSDF specification of channelizer subsystem. 12

Case Study of Channelizer/receiver Application (3) Average processing power of all the available configurations. • Platform for Channelizer: Raspberry Pi 3 Model B • Device for Power Measurement: Tektronix Keithley Series 2280 Precision Measurement DC Power Supply 13

Case Study of Channelizer/receiver Application (4) Simulation results for MDP-I. Comparison among MDP-generated policies and fixed-configuration designs. In addition to providing better energy efficiency compared to the fixed configuration designs, our MDP approach • can be configured systematically to generate a much larger set of trade-off options (Pareto-optimized fronts) • ensures optimality (with respect to the given reward function) 14

• Solver running time – MDP Solver: MATLAB-based open source solver, named MDPSOLVE – Platform for MDP Solver: • Processor: i 7 -4710 HQ running at 2. 50 GHz • RAM: 12. 0 GB • OS: 64 -bit Windows 10 – – MDP-I solver running time: 294 ms MDP-II-a solver running time: 50. 8 ms MDP-II-b solver running time: 41. 5 ms In a deployment with a fixed processing system (MDP-II-b) and changing external environment (MDP-II-a), the hierarchical MDP scheme reduces the solver time from 294 ms to 50. 8 ms, which is a factor of over 5. 7 X smaller. • Model size – MDP-I: 1. 63 MB – MDP-II: 265 k. B – MDP-II reduces model size by a factor of over 6. 1 X. 15

Conclusions • We propose the HMCSM framework for design and implementation of adaptive embedded signal processing systems. • HMCSM stands for Hierarchical MDP framework for Compact System-level Modeling. • HMCSM provides a structured methodology that integrates dataflow methods; MDP formulation using compact, hierarchical models; optimal policy generation at design time; and dynamic, system-level reconfiguration at run time. • We demonstrate the effectiveness of our new MDPbased system design framework through experiments with an adaptive wireless communications receiver. 16

Thanks! 17