Distributed Computing with Python Carlos Varela Rensselaer Polytechnic

Distributed Computing with Python Carlos Varela Rensselaer Polytechnic Institute C. Varela, RPI 1

Distributed Computing with Python • Using standard TCP/IP (or UDP) sockets • Using high-level libraries for remote method invocation – XML-RPC – Pyro (PYthon Remote Objects) C. Varela, RPI 2

Networking with Python Sockets • • • Resembles C socket API Can use UDP (datagrams) or TCP (streamed connections) Does not require “non-standard” libraries Low-level, yet relatively easy to use and portable Enables more efficient communication and is programming language independent. • New features require protocol extensions C. Varela, RPI 3

�Sockets: An Echo Server� from socket import * def main(): s = socket(AF_INET, SOCK_STREAM) s. bind((‘’, 50007)) # port number s. listen(1) conn, (remotehost, remoteport) = s. accept() print 'connected by', remotehost, remoteport while 1: data = conn. recv(1024) # buffer size if not data: break conn. send(data) conn. close() main() C. Varela, RPI 4

�Sockets: An Echo Client� from socket import * def main(): s = socket(AF_INET, SOCK_STREAM) s. connect((‘localhost’, 50007)) # port number s. send(‘Hello World’) data = s. recv(1024) # buffer size s. close() print data main() C. Varela, RPI 5

Remote Procedure Calls with Python XML-RPC library • XML-RPC is programming-language/O. S. independent • Uses XML as message format for parameters and return values, e. g. : <params> <param> <value><string>Hello World</string></value> </params> • Uses HTTP as the transport protocol. • Enabling technology for building “web services”. • Example requires downloading xmlrpcserver. py library: http: //svn. python. org/view/python/trunk/Demo/xmlrpc/ C. Varela, RPI 6

�XML-RPC: An Echo Server� import Socket. Server; import xmlrpcserver; import xmlrpclib class Echo. Req. Handler(xmlrpcserver. Request. Handler): def call(self, method, params): print "Dispatching: ", method, params try: server_method = getattr(self, method) except: raise Attribute. Error, ”No method: %s" % method return server_method(params) def echo(self, value): return xmlrpclib. dumps(value) server = Socket. Server. TCPServer(('', 8000), Echo. Req. Handler) 7 server. serve_forever() C. Varela, RPI

�XML-RPC: An Echo Client� import xmlrpclib echosvr = xmlrpclib. Server("http: //localhost: 8000") print echosvr. echo("Hello World") print echosvr. echo(10) C. Varela, RPI 8

Distributed Computing with Python Remote Objects (PYRO) • Resembles Java RMI: Server Daemon, Client Proxy • To find remote objects: – By name, e. g. , “PYRONAME: //echo”, requires name server – By location, e. g. , “PYROLOC: //localhost: 7766/echo” – By URI, e. g. , “PYRO: //127. 0. 0. 1: 7766/7 f 0000010512127841288 b 1892 e 90 a 71” • Method arguments, return values, and exceptions marshalled (“pickled”) over the network • Example code requires downloading and installing Pyro at: http: //pyro. sourceforge. net/ C. Varela, RPI 9

�Pyro: An Echo Server� import Pyro. core class Echo(Pyro. core. Obj. Base): def __init__(self): Pyro. core. Obj. Base. __init__(self) def echo(self, string): return string; Pyro. core. init. Server() daemon=Pyro. core. Daemon() uri=daemon. connect(Echo(), "echo-object") print "The object's uri is: ", uri daemon. request. Loop() C. Varela, RPI 10

�Pyro: An Echo Client� import Pyro. core echo. Proxy = Pyro. core. get. Proxy. For. URI( "PYROLOC: //localhost: 7766/echo-object") print echo. Proxy. echo("Hello World") C. Varela, RPI 11

Python Remote Objects (PYRO) • Python programming language-dependent • Event server for publish-subscribe communication • Mobile code, but BEWARE: No security guarantees! C. Varela, RPI 12

Exercises 1. Compare the round trip time of the Echo application using: • • • TCP/IP XML-RPC Pyro 2. Use the XML-RPC client in Python to connect to an existing web service (e. g. , Google) • http: //www. xmlrpc. com/google. Gateway 3. Modify the echo example in Pyro to use the name service. What are alternative ways to get to the name server? 4. Does Pyro support distributed garbage collection? C. Varela, RPI 13