Design Patterns in TclTk Brent Welch welchscriptics com

Design Patterns in Tcl/Tk Brent Welch <welch@scriptics. com> http: //www. scriptics. com/people/brent. welch Tcl/Tk Design Patterns Tcl-1

Themes l Tcl Architecture Background » Scripting vs. Systems Programming l Design Patterns and Sample Apps » “Micro” patterns for coding techniques » “Macro” patterns for application structure Tcl/Tk Design Patterns Tcl-2

Tcl Design Patterns l l l l “Bricks and Glue” Script callbacks “Glue and GUI” for UNIX programs Cooperating applications and Safe-Tcl Control logic for data processing channels Device control and test environments Middle tier applications Web-based applications Tcl/Tk Design Patterns Tcl-3

Tcl is two things: l Tcl is a scripting language » Interpreted for rapid turnaround » Variables, procedures, error handlers » exec programs, file I/O, network sockets, . . . l Tcl is a C library » Easy to add to applications » Applications define Tcl commands that are implemented in C, C++, or Java Tcl/Tk Design Patterns Tcl-4

When to Script? l Systems Programming (C, C++, Java) » Strong types, Static interfaces l Scripting (Tcl, Perl, Python, Basic, . . . ) » Dynamic types, Fluid interfaces l Scripting Architecture » The art of defining the “sweet spot” between scripts and components Tcl/Tk Design Patterns Tcl-5

The First Pattern l Components are Commands » Implemented in C, C++, Java, or Tcl » Take string arguments, return a string* l Language constructs such as if, proc, while, and foreach are commands l Command-base approach eliminates a lot of syntax found in other languages Tcl/Tk Design Patterns Tcl-6

Built-in Tcl Commands l l l l Variables Control-Flow Procedures Namespaces Strings Pattern Matching Lists and Arrays Introspection Tcl/Tk Design Patterns l l l l Exec Programs File System Input / Output Clock and Timers Network Sockets Safe-Tcl Tk GUI Toolkit Various Extensions. . . Tcl-7

“Bricks and Glue” l Commands are like (fancy) Bricks » They are specialized and efficient » They have configuration knobs and dials l Scripts are Glue that assembles bricks » Scripts set configuration parameters » Scripts define behaviors (callbacks) » Scripts sequence actions Tcl/Tk Design Patterns Tcl-8

Application Structure “glue” Scripts Callbacks Tcl interpreter Components Tcl/Tk Design Patterns “bricks” Tcl-9

Command Callbacks l Building blocks call out to Tcl scripts at important times to determine behavior » I/O channel read and write handlers » Server socket accept handlers » Variable traces » Tk buttons » SQL Iterator Scripts Tcl interpreter Tcl/Tk Design Patterns Tcl-10

Pure Script Pattern l 100% Pure Tcl Applications » Cross platform » Rapid Development l Not the original design center for Tcl! » Namespaces, multiple interpreters, threading added for structure l Sample Applications: » HTML editor (webtk), web server (tclhttpd) Tcl/Tk Design Patterns Tcl-11

“Glue and GUI” Layer Tcl scripts and Tk GUI on top of existing applications l System administration tools, but with nice GUI l » exec command launches program, returns the standard output as value » open pipelines with file redirection l Not as efficient as built-in commands Tcl/Tk Design Patterns Tcl-12

Expect l Automatic control of interactive applications. » exp_send - generate input » expect - pattern match on output. expect { login: {exp_send “welchrn”; continue} *assword: {exp_send “a secretrn”} timeout {puts “Remote system is down”} } Tcl/Tk Design Patterns Tcl-13

Expect Applications Automate FTP, Telnet, S/Key l Set password on 150 remote systems l “Screen scrape” mainframe (3270) apps l Automatic test frameworks l Play two copies of chess against each other l Tcl/Tk Design Patterns Tcl-14

“Glue and GUI” (exmh) l MH applications, Tcl/Tk user interface l www. beedub. com/exmh Unread Mail Tcl/Tk C prog GUI MH email Tcl/Tk Design Patterns MIME Glimpse Search Msg Filters Bounces URL scan PGP Ispell User Procs Faces Tcl-15

Extensible Applications Library facility makes it easy to add user code to an application l Introspection makes hook points easy to implement l » Choosing hook points is an art l X resource database trick for extensible buttons and menus Tcl/Tk Design Patterns Tcl-16

Introspection l l Example: hooks for extensibility. Just define extension procedures: proc Edit. Hook 1 {widget file} {. . . } proc Edit. Hook 2 {widget file} {. . . } l In application, look for and invoke procedures. foreach proc [info procs Edit. Hook*] { $proc $widget $file } Tcl/Tk Design Patterns Tcl-17

User-Defined Buttons, Menus X resources store widget attributes, but have limitations (i. e. , we have to hack a bit) l Non-standard resources record buttons, menu buttons, and menu entries l Standard resources define button comands, text, variables l Non-standard resource define menu entry commands, text, cascades l Tcl/Tk Design Patterns Tcl-18

Exmh l Core User Interface » MH Mail programs » Hook points and user script library l Community Contributions » » » Glimpse full text search PGP authentication Faces glyph database Ispelling checker Url Scanning Tcl/Tk Design Patterns Tcl-19

Model Sim l Leading CAD Simulation Tool from Model Technologies » www. model. com Application GUI built in Tcl/Tk l Customers create custom control panels for their simulations l Tcl/Tk Design Patterns Tcl-20

Control for Data Processing l Control Logic vs. Data Processing » Tcl for setup and configuration » C, C++, Java for data processing modules l Application-specific framework for processing modules » Audio / Video / Image Processing » Simulation Frameworks Tcl/Tk Design Patterns Tcl-21

Image / Audio Processing Tcl Control Logic, TK GUI Data Source Data Filters CD compression - try many parameter permutations each night Tcl/Tk Design Patterns Data Sink Tcl-22

Network Servers l Tcl’s Socket and I/O subsystem make this easy » Callbacks to handle connections » Event-driven I/O to multiplex the server l Script-based servers are robust » Tcl doesn’t leak memory » Script errors do not “dump core” Tcl/Tk Design Patterns Tcl-23

Socket Setup l Client socket s [socket $host $port] l Server socket proc Accept {newsock ipaddr port} { # New connection in newsock fileevent $newsock readable [list Read $newsock] } socket -server Accept $port vwait forever Tcl/Tk Design Patterns Tcl-24

Event Driven I/O l Wait for I/O with fileevent $sock readable [list Read $sock] l Read one line or block per event proc Read {sock} { if {[eof $sock]} { close $sock ; # clears fileevent } else { gets $sock line } } Tcl/Tk Design Patterns Tcl-25

Desktop Agent l Shamancorp. com desktop agent » Tcl for Windows, Mac portability » Small footprint compared to competitors l Agent performs software inventory, allows distribution of updates, including update of the agent itself Tcl/Tk Design Patterns Tcl-26

Cooperating Applications l Network Managemnt Applications » Nations. Bank Agent implementations l What about trust? l » Safe-Tcl limits capabilities Tcl/Tk Design Patterns Tcl-28

Safe-Tcl l l Dual interpreters (like kernel/user Unrestricted Restricted space). Interpreter(s) Unrestricted interpreter (i. e. , kernel) for user and receiving application. Incoming Tcl script executes in restricted interpreter (i. e. , user space): Aliases no dangerous commands. Aliases from restricted interpreter back to unrestricted interpreter (i. e. , kernel calls). Plug-in for Netscape and Explorer Tcl/Tk Design Patterns Tcl-29

Custom Control Systems l High Energy Physics, Oil Drill Rig » One of a kind machinery » Device drivers » Database » Control logic » Tk GUI » 1 -200 KLines, 50% C, 50% Tcl/Tk Design Patterns Tcl-30

Device Under Test Table-Driven Tk GUI Tcl Test Suite Serial Interface Family of Telecom Line Card Controll Chips Tcl/Tk Design Patterns ASIC V 1 ASIC V 2 ASIC V 3 Tcl-31

Device Under Test (AMD) Version 1, completly custom DOS testing application l Version 2, with more on the way l Tcl-based table-driven approach: l » Describe serial communication interface » Describe processor instructions » Auto-gen Tk GUI for test engineer » Tcl script test suite Tcl/Tk Design Patterns Tcl-32

Middle-Tier Application Vignette CNET AOL Scriptics Web Browser App Server Tcl SQL Database Tcl/Tk Design Patterns Tcl-33

Tcl. Httpd Web Server 250 -line “mini server” l 7000 line full featured server l Points of extensibility l » URL subtree – Application direct URL » Document handlers – Tcl+HTML Templates » Authentication schemes Tcl/Tk Design Patterns Tcl-34

Application Direct URL l Introspection for URL hook » Tcl Procedure name corresponds to URL » Form data mapped automatically onto Tcl procedure arguments l Return value of procedure is the HTML content returned to the browser Tcl/Tk Design Patterns Tcl-35

Application Direct URL Direct_URL Debug /debug proc Debug/echo {args} { set html <dl> foreach {name val} $args { append html “<dt>$name<dd>$val” } append html </dl> return $html } http: //www. com/debug/echo? a=b&c=d Tcl/Tk Design Patterns Tcl-36

Web Page Generation form data Template Page Application Code Tcl/Tk Design Patterns HTTP Request cache HTML Page Tcl-37

Tcl+HTML Templates l Tcl embedded in HTML expanded by the server (i. e. , the subst command) Shared files in each directory contain code, global settings l Server can cache result in HTML file, only re-gen when template or shared files change l Tcl/Tk Design Patterns Tcl-38

Site-Wide Look [Page. Head “The title” “some keywords”] [Main. Menu Support FAQ] HTML content [Page. Footer] l A few lines of Tcl expand into hundreds of lines of HTML to create navigational structure around page content Tcl/Tk Design Patterns Tcl-39

XML “Templates” <Page title=“Foo” keywords=“Bar”/> <Main. Menu lev 1=“Support” lev 2=“FAQ”/> HTML content </Page> Convert XML to HTML as needed Tcl/Tk Design Patterns Tcl-40

Scriptics Web Site Global Look and feel through templates l Tcl Resource Center. Annotated index of 500+ Tcl-related URLs l On-line visitor database l Web store front. Credit card service, fulfillment service l Email pipe into Scriptics database l Tcl/Tk Design Patterns Tcl-41

Appendix l The remaining slides show » Where to get Tcl/Tk » Some suggested books » Some low-level coding patterns and examples Tcl/Tk Design Patterns Tcl-42

Getting Started Downloading Tcl and Tk http: //www. scriptics. com/software/choose. html Guide to which release is best for you. ftp: //ftp. scriptics. com/pub/tcl Primary FTP site Tcl Shell Programs wish, Tcl and Tk GUI toolkit tclsh, Tcl only Tcl/Tk Design Patterns Tcl-43

Documentation Practical Programming in Tcl and Tk, 2 nd Ed. http: //www. beedub. com/book/ ~20 books on the market On-line manual pages http: //www. scriptics. com/man/ Unix "man" command, Tk. Man Windows help files Macintosh HTML folder Tcl/Tk Design Patterns Tcl-44

Tcl Books http: //www. scriptics. com/resource/doc/books/ Practical Programming in Tcl and Tk, Welch Graphical Applications in Tcl/Tk, Johnson Tcl/Tk Tools, Harrison et. al. Effective Tcl/Tk Programming, Mc. Lennan, Harrison Tcl/Tk for Programmers, Zimmer Tcl/Tk for Real Programmers, Flynt Tcl and the Tk Toolkit, Ousterhout Tcl for Dummies, Webster, Francis Exploring Expect, Libes Tcl/Tk Design Patterns Tcl-45

Saving Data As Tcl Scripts foreach name [info globals] { upvar #0 $name var if [array exists var] { puts [list array set $name [array get var]] } else { puts [list set $name $var] } } Tcl/Tk Design Patterns Tcl-46

Efficient String Processing l Convert input into a Tcl program » Protect Tcl special characters » regsub -all to re-write data into a Tcl program » eval or subst to apply commands to the data l Example: decoding %xx in a URL regsub -all {%([0 -9 a-f. A-F])} $url {[format %c 0 x1]} url set url [subst $url] Tcl/Tk Design Patterns Tcl-47

Subst vs. Eval after the Regsub l subst: only process data matched by regsub » Html. Decode. Entity example in paper < © &169; » quote $, [, and » rewrite into the following general form: unmatched data [command matched data] l eval: process both matched and unmatched data » Html_Parse example in paper » convert braces and backslashes into HTML entities » rewrite into the following general form: } command matched data { unmatched data Tcl/Tk Design Patterns Tcl-48

HTML_Parse example HTML input <title>Tcl { syntax }</title> This is an image. <img src=foo. gif> Tcl program func func {start} {} {title} {} {} {Tcl &ob; syntax &cb; } {title} {/} {} {This is an image. } {img} {} {src=foo. gif} {} {start} {/} {} {} Tcl/Tk Design Patterns Tcl-49

Multiple Interpreters l Interpeters have path names (lists) » {} is self. {a b} is a child of {a} interp create ? -safe? path eval command args interp delete path l Safe interpreters start with a limited set of commands. E. g. , no open or exec. Tcl/Tk Design Patterns Tcl-50

Tcl as a C library interp = Tcl_Create. Interp(); Tcl_Create. Command(interp, “foo”, Foo_Cmd, foo. Data); Tcl_Set. Var(interp, “foo_version”, “ 1. 0”); Tcl_Eval(interp, “source myfile. tcl”); Tcl_Set. Result(interp, “the answer”); Tcl_Link. Var(interp, “x”, &value, TCL_LINK_INT); Tcl/Tk Design Patterns Tcl-51

C Command Procedure int Foo_Cmd(interp, data, argc, argv) Tcl_Interp *interp; /* Script context */ Client. Data data; /* Private data */ int argc; char *argv[]; /* array of string arguments */ Very similar to a main program interface l Interface to script is string-oriented l Tcl/Tk Design Patterns Tcl-52

Tcl 8. 0 Data Interface int Foo_Cmd(interp, data, objc, objv) Tcl_Interp *interp; Client. Data data; int argc; Tcl_Obj *objv[]; l /* Script context */ /* Private data */ /* array of objects*/ Tcl_Obj has string rep and native rep Tcl/Tk Design Patterns Tcl-53

Tcl_Obj - Dual Ported Value l Native representation and a string » String rep computed lazily l Built-in object types: » integer, double, boolean, list, array, bytecode, compiled regular expression l New type managers can be added » Java object reference (Tcl. Blend, JACL) » Active X object Tcl/Tk Design Patterns Tcl-54

Tcl_Obj struct typedef struct Tcl_Obj { int refcount; char *bytes; int length; Tcl_Obj. Type *type. Ptr; union { long. Value; double. Value; VOID *other. Value. Ptr; struct { VOID *ptr 1; VOID *ptr 2; } two. Ptr. Value; } internal. Rep; } Tcl_Obj; Tcl/Tk Design Patterns typedef struct Tcl_Obj. Type { char *name; Proc free. Int. Rep. Proc; Proc dup. Int. Rep. Proc; Proc update. String. Proc; Proc set. From. Any. Proc; Proc get. Interface; } Reference counts String representation Native representation Type information Tcl-55