Concurrent Functional Programming with Erlang and OTP Open

Concurrent Functional Programming with Erlang and OTP (Open Telecom Platform) Bjarne Däcker <bjarne@erix. ericsson. se> Computer Science Laboratory Ericsson Utvecklings AB Acknowledgements Thomas Arts <thomas@erix. ericsson. se> Hans Nilsson <hans@erix. ericsson. se> Torbjörn Keisu <torbjorn. keisu@ericsson. com> Ulf Wiger <ulf. wiger@ericsson. com> 2001 -03 -19 ETX/D/XPS-01: 002 Uen 1

The setting l 1995 PC Week Study of software projects: – 16% successful – 53% operational (but less than successful) – 31% cancelled l Butler Group 1997 on large software projects – 5 out of 6 large projects fail – In >$300 M companies, 9 out of 10 large projects fail l How to approach this? – Use high-level modeling tools & generate code? – Raise the level of programming language? – Fight all causes for project failure! 2001 -03 -19 ETX/D/XPS-01: 002 Uen 2

Telecom industry l Switches, routers, base-stations l Networks l Mobile telephones 2001 -03 -19 ETX/D/XPS-01: 002 Uen 3

Requirements on a Programming Technology for Telecommunication Switching Systems l Massive concurrency l Soft realtime l Distribution l Interaction with hardware l Very large software systems l Complex functionality l Continuous operation for many years l Software maintenance without stopping the system l Stringent quality and reliability requirements l Fault tolerance errors 2001 -03 -19 ETX/D/XPS-01: 002 Uen 4

History of Erlang 1984: Ericsson Computer Science Lab formed 1984 -86: Experiments programming POTS with several languages No language well suited for telecom systems development 1998: Open Source Erlang 1991: First fast implementation 1987: Early Erlang Prototype projects 1996: Open Telecom Platform (research on verification. . . ) 1993: Distributed Erlang 1995: Several new projects 2001 -03 -19 ETX/D/XPS-01: 002 Uen 5

The Ancestry of Erlang Concurrent systems programming languages like Ada, Modula or Chill Functional programming languages like ML or Miranda Concurrent functional programming language Erlang 2001 -03 -19 ETX/D/XPS-01: 002 Uen 6

Erlang Highlights l Declarative l Concurrency l Soft real-time l Robustness l Distribution l Hot code loading l External interfaces l Portability Functional programming language High abstraction level Pattern matching Concise readable programs 2001 -03 -19 ETX/D/XPS-01: 002 Uen 7

Erlang Example Basics - Factorial function Definition 1 Implementation n=0 n! = n*(n-1)! n 1 -module(ex 1). -export([factorial/1]). factorial(0) -> 1; factorial(N) when N >= 1 -> N * factorial(N-1). Eshell V 5. 0. 1 (abort with ^G) 1> c(ex 1). {ok, ex 1} 2> ex 1: factorial(6). 720 2001 -03 -19 ETX/D/XPS-01: 002 Uen 8

Erlang Example A few very high-level constructs - Quick. Sort -module(ex 2). -export([qsort/1]). qsort([Head|Tail]) -> First = qsort([X || X <- Tail, X =< Head]), Last = qsort([Y || Y <- Tail, Y > Head]), First ++ [Head|Last]; qsort([]) -> []. Eshell V 5. 0. 1 (abort with ^G) 1> c(ex 2). {ok, ex 2} 2> ex 2: qsort([7, 5, 3, 8, 1]). [1, 3, 5, 7, 8] "all objects Y taken from the list Tail, where Y > Head" 2001 -03 -19 ETX/D/XPS-01: 002 Uen 9

Erlang Highlights l Declarative l Concurrency l Soft real-time l Robustness l Distribution l Hot code loading l External interfaces l Portability Either transparent or explicit concurrency Light-weight processes Highly scalable 2001 -03 -19 ETX/D/XPS-01: 002 Uen 10

Erlang Example Creating a new process using spawn -module(ex 3). -export([activity/3]). activity(Name, Pos, Size) -> ………… Pid = spawn(ex 3, activity, [Joe, 75, 1024]) activity(Joe, 75, 1024) 2001 -03 -19 ETX/D/XPS-01: 002 Uen 11

Erlang Example Processes communicate by asynchronous message passing Pid ! {data, 12, 13} receive {start} -> ……… {stop} -> ……… {data, X, Y} -> ……… end 2001 -03 -19 ETX/D/XPS-01: 002 Uen 12

Erlang Example Concurrency - Finite State Machine ringing_a(A, B) -> Selective receive {A, on_hook} -> back_to_idle(A, B); {B, answered} -> Asynchronous send A ! {stop_tone, ring}, switch ! {connect, A, B}, conversation_a(A, B) after 30000 -> Optional timeout back_to_idle(A, B) end. back_to_idle(A, B) -> A ! {stop_tone, ring}, B ! terminate, idle(A). 2001 -03 -19 ETX/D/XPS-01: 002 Uen 13

Erlang Highlights l Declarative l Concurrency l Soft real-time l Robustness l Distribution l Hot code loading l External interfaces l Portability Response times in the order of milliseconds Per-process garbage collection 2001 -03 -19 ETX/D/XPS-01: 002 Uen 14

Erlang Highlights l Declarative l Concurrency l Soft real-time l Robustness l Distribution l Hot code loading l External interfaces l Portability Simple and consistent error recovery Supervision hierarchies "Program for the correct case" 2001 -03 -19 ETX/D/XPS-01: 002 Uen 15

Erlang Example Cooperating processes may be linked together using spawn_link(…, …, …) or link(Pid) 2001 -03 -19 ETX/D/XPS-01: 002 Uen 16

Erlang Example When a process terminates, an exit signal is sent to all linked processes … and the termination is propagated 2001 -03 -19 ETX/D/XPS-01: 002 Uen 17

Erlang Example Exit signals can be trapped and received as messages receive {‘EXIT’, Pid, . . . } ->. . . end 2001 -03 -19 ETX/D/XPS-01: 002 Uen 18

Erlang Example Robust systems can be built by layering “Supervisors” “Workers” 2001 -03 -19 ETX/D/XPS-01: 002 Uen 19

Erlang Highlights l Declarative l Concurrency l Soft real-time l Robustness l Distribution l Hot code loading l External interfaces l Portability Explicit or transparent distribution Network-aware runtime system 2001 -03 -19 ETX/D/XPS-01: 002 Uen 20

Transparent Distribution B ! Msg C ! Msg Erlang Run-Time System network 2001 -03 -19 ETX/D/XPS-01: 002 Uen 21

Simple RPC {rex, Node} ! {self(), {apply, M, F, A}}, receive {rex, Node, What} -> What end loop() -> receive {From, {apply, M, F, A}} -> Answer = apply(M, F, A), From ! {rex, node(), Answer} loop(); _Other -> loop() end. 2001 -03 -19 ETX/D/XPS-01: 002 Uen 22

Erlang Highlights l Declarative l Concurrency l Soft real-time l Robustness l Distribution l Hot code loading l External interfaces l Portability Easily change code in a running system Enables non-stop operation Simplifies testing 2001 -03 -19 ETX/D/XPS-01: 002 Uen 23

Erlang Example Version 1 Version 2 2001 -03 -19 ETX/D/XPS-01: 002 Uen 24

Erlang Highlights l Declarative l Concurrency l Soft real-time l Robustness l Distribution l Hot code loading l External interfaces l Portability "Ports" to the outside world behave as Erlang processes 2001 -03 -19 ETX/D/XPS-01: 002 Uen 25

Erlang Example Port External process Port ! {self(), {command, [1, 2, 3]}} 2001 -03 -19 ETX/D/XPS-01: 002 Uen 26

Erlang Example Port External process receive {Port, {data, Info}} -> end 2001 -03 -19 ETX/D/XPS-01: 002 Uen 27

Erlang Highlights l Declarative l Concurrency l Soft real-time l Robustness l Distribution l Hot code loading l External interfaces l Portability Erlang runs on any UNIX, Windows, Vx. Works, . . . Supports heterogeneous networks 2001 -03 -19 ETX/D/XPS-01: 002 Uen 28

Systems Overview Applications written in Erlang Applications written in C, C++ or Java OTP Components Standard Libraries Erlang Run-Time System Hardware and Operating System 2001 -03 -19 ETX/D/XPS-01: 002 Uen 29

Erlang/OTP l Open Telecom Platform l Middleware for Erlang development l Designed for fault tolerance and portability l Behaviors: A formalization of design patterns l Components – – – – Error handling, reporting and logging Mnesia, distributed real-time database management system CORBA IDL Compiler, Java & C Interface Support HTTP Server SNMP Agent. . . 2001 -03 -19 ETX/D/XPS-01: 002 Uen 30

OTP Behaviors l "A formalization of design patterns" – A behavior is a framework + generic code to solve a common problem – Each behavior has built-in support for debugging and software upgrade – Makes it easier to reason about the behavior of a program l Examples of OTP behaviors – – – applicationdefines how an application is implemented supervisor used to write fault-tolerant supervision trees gen_server for writing client-server applications gen_event for writing event handlers gen_fsm for finite state machine programming 2001 -03 -19 ETX/D/XPS-01: 002 Uen 31

The standard textbook Joe Armstrong, Robert Virding, Claes Wikström and Mike Williams, Concurrent Programming in Erlang, Prentice Hall, 1996, 2 nd edition, ISBN 0 -13 -508301 -X 2001 -03 -19 ETX/D/XPS-01: 002 Uen 32

Interesting web pages Open source Erlang www. erlang. org Commercial Erlang www. erlang. se French Erlang site www. erlang-fr. org 2001 -03 -19 ETX/D/XPS-01: 002 Uen 33

Courses/year (10 -15 pupils/course) 2001 -03 -19 ETX/D/XPS-01: 002 Uen 34

Requests/month to www. erlang. org 2001 -03 -19 ETX/D/XPS-01: 002 Uen 35

Downloads/month from www. erlang. org 2001 -03 -19 ETX/D/XPS-01: 002 Uen 36

l AXD 301: A Telephony-Class, scalable (10 -160 GBps) ATM switch designed from scratch in less than 3 years l AXD 301 Success factors: – – Highly pragmatic, holistic approach Competent organisation Efficient process Excellent technology (e. g. Erlang/OTP) l More than just technology. . . – Consider all factors together from the start – Erlang was a perfect match for our approach 2001 -03 -19 ETX/D/XPS-01: 002 Uen 37

AXD 301 in the marketplace l Central component in Ericsson's ENGINE offering ENGINE: Migrating today's vertical networks into a single multi-service backbone l Several major operators – – – British Telecom Vodaphone Worldcom Telia Diveo. . . 2001 -03 -19 ETX/D/XPS-01: 002 Uen 38

Briefly about the term Carrier-Class l To us, "Carrier-Class", "Telephony-Class" and "Telecom Profile" are synonymous l The quality we've come to expect from public telephony networks l The trend towards multimedia services requires Carrier-Class in more systems l More than just duplication of hardware: – – – Fault-tolerant software In-service hardware expansion In-service software upgrade Load tolerance Flexibility (frequent changes + long service life) There's no such thing as "almost Carrier-Class"! l Target: 99, 999% ("five nines") availability, including planned outages 2001 -03 -19 ETX/D/XPS-01: 002 Uen 39

Telecom-Class System Architecture simple wire-speed logic complex soft-real-time logic Control Processors Line ATM Termination ATM ATB CE ATB FR ATB Switch Core CP IO Optional Processors Server Device L 3 F Device Processor on Each Board Mandatory Mated Processor Pair ATB CP IO Clock & Synchronization 2001 -03 -19 ETX/D/XPS-01: 002 Uen 40

Programming languages (control system) l Erlang: ca 1 million lines of code – Nearly all the complex control logic – Operation & Maintenance – Web server and runtime HTML/Java. Script generation l C/C++: ca 500 k lines of code – Third party software – Low-level protocol drivers – Device drivers l Java: ca 13 k lines of code – Operator GUI applets 2001 -03 -19 ETX/D/XPS-01: 002 Uen 41

Experiences from AXD 301 SW Design l Using Erlang in Complex Systems – – – Fits very well with the incremental design method High programmer satisfaction Outstanding support for robustness and concurrency Very few side-effects easier to add/change single components Small directed teams can achieve impressive results l Productivity estimates – Similar line/hour programmer productivity – 4 -10 fewer lines of source code (compared to C/C++, Java, PLEX) l 4 -10 x higher programmer productivity – Similar number of faults per 1000 lines of source code l 4 -10 x higher quality 2001 -03 -19 ETX/D/XPS-01: 002 Uen 42

l Functional Requirements – Use cases (in telecom “traffic cases”) – User interfaces –. . . l Non-functional Requirements – – – Code updating without interrupting the service Distribution over several processors Automatic handover upon error Limited restart time. . . l The non-functional requirements are often much trickier to handle and require technology bottom-up rather than analysis top-down. 2001 -03 -19 ETX/D/XPS-01: 002 Uen 43

Efficient Program Development Requirements l Interaction with the real environment Ideas Prototyping l Powerful and appropriate abstraction mechanisms l Efficient implementation l Useful tools Productification 2001 -03 -19 ETX/D/XPS-01: 002 Uen 44

A Simple Erlang-XML Document XML <? xml version=“ 1. 0”? > <home. page title=“My Home Page”> <title> Welcome to My Home Page </title> <text> <para> Sorry, this home page is still under construction. Please come back soon! </para> </text> </home. page> Erlang {‘home. page’, [{title, “My Home Page”}], [{title, “Welcome to My Home Page”}, {text, [{para, “Sorry, this home page is still under ” “construction. Please come back soon!”} ]} ]}. Almost equivalent 2001 -03 -19 ETX/D/XPS-01: 002 Uen 45

Erlang Summary l Declarative – Compact code l Concurrency – Light-weight processes – Message passing l Soft real-time l Robustness www. erlang. org www. erlang. se – Process supervision – Error trapping l Distribution l Hot code loading l External interfaces – To hardware and other languages l Portability 2001 -03 -19 ETX/D/XPS-01: 002 Uen 46

The Research Continues. . . l Hi. PE - High Performance Erlang (Uppsala, Astec) l ETOS - Erlang to Scheme (Montréal) l Erlang Verification (SICS, Astec) l Type System (Glasgow) l “Safe” Erlang (Canberra) l Specification Techniques (Aachen) l Erlang Processor (Ericsson CADLab) l. . . 2001 -03 -19 ETX/D/XPS-01: 002 Uen 47

Erlang User Conference 2000 Hurray !!! 2001 -03 -19 ETX/D/XPS-01: 002 Uen 48

Welcome to the next EUC September 27, 2001 Älvsjö. Stockholm www. erlang. se Combined with IFL - International Workshop on the Implementation of Functional Languages September 24 -26, 2001 Älvsjö. Stockholm 2001 -03 -19 ETX/D/XPS-01: 002 Uen 49