Source Code Analysis Using BAT Reverse Engineering Source

Source Code Analysis Using BAT Reverse Engineering (Source Code Analysis) © SERG

What is Static Analysis? • Mining source code for information. • Using that information to present abstractions of, and answer questions about, software structure. Reverse Engineering (Source Code Analysis) © SERG

What can we get from source code analysis? • Type of information is model dependent – In almost any language, we can find out information about variable usage: Who? Where? etc. – In an OO environment, we can find out which classes use other classes, which are a base of an inheritance structure, etc. – We can also find potential blocks of code that can never be executed in running the program (dead code). Reverse Engineering (Source Code Analysis) © SERG

BAT • Is a tool that lets us perform static analysis on Java programs (class files). – Builds an XML database of entities and relationships in a system. – Can use several tools for querying and visualizing the data. Reverse Engineering (Source Code Analysis) © SERG

Entities • ‘Entities’ are individuals that live in the system, and attributes associated with them. Some examples: – Classes, along with information about their superclass, their scope, and ‘where’ in the code they exists. – Methods/functions and what their return type or parameter list is, etc. – Variables and what their types are, and whether or not they are static, etc. Reverse Engineering (Source Code Analysis) © SERG

Relationships • ‘Relationships’ are interactions between the entities in the system. Relationships include: – Classes inheriting from one another. – Methods in one class calling the methods of another class, and methods within the same class calling one another. – One variable referencing another variable. Reverse Engineering (Source Code Analysis) © SERG

Creating BAT Databases • BAT is really a library that can process JAR files • BATAnalyzer is a small app wrapped around BAT to return a full XML database from BAT for later processing Need to give – Found at: BATROOT/analyzer/src • To run: Java a lot of Memory to process large projects export PATH=/usr/remote/serg/jdk 1. 5. 0_11/bin/: $PATH java -Xmx 2 G -cp /usr/remote/serg/binbat 2 toxml. jar: /usr/remote/serg/bin/batanalyzer. jar batanalyzer. Main <JAR> <OUTPUT> Project to analyze XML output file BAT API Reverse Engineering (Source Code Analysis) Call to analyzer © SERG

Provided Tools to deal with BAT • bdef – A BASH wrapper around XSLT queries to get entity information • bref – A BASH wrapper around XSLT queries to get relationship information • dot – A visualization tool. Takes information from query and displays it as a graph. • On TUX to get the scripts do: export PATH=$PATH: /usr/remote/serg/bin/ Reverse Engineering (Source Code Analysis) © SERG

bdef Syntax • bdef takes information from the entities databased on a query, and returns the results in an ascii-table. bdef xml_file entity_kind entity_name [attr=val] – – xml_file is the xml file containing the extracted database entity_kind is the ‘type’ of entity to retrieve. entity_name is a pattern to match for names of entities. attr=val are bindings to match for attributes of the entity Reverse Engineering (Source Code Analysis) © SERG

Entity Kinds • Chava recognizes several types of entity ‘kinds’ for use in the bdef/bref commands. • • m c f - is for Method is for Class is for Field is a match for any entity_kind Reverse Engineering (Source Code Analysis) © SERG

Entity Names • An entity name can assume many forms following reg. EX patterns – Explicit name (e. g. , ‘my. Temp. String. Var’) – Wild-card Pattern (e. g. , ‘my. Temp. *’) – A complete wild-card, denoted with ‘. *’ Reverse Engineering (Source Code Analysis) © SERG

Attribute=Value settings are used to further restrict a query based on some condition specified as reg. EX. • Any field is searchable • The most common restriction is to restrict to a specific file, or to filter out a file. E. g. , • Attribute=Value bdef file. xml - - filename=File. IDo. Like. java bdef file. xml - - filename=[^(File. IDo. NOTLike. java)] Reverse Engineering (Source Code Analysis) © SERG

Fields • Class – name, filename, scope, deprecated, final, abstract • Method – name, class, filename, scope, static, deprecated, final, abstract, varargs, bridge, native, synchronized, return, parameters • Field – Name, class, filename, type, scope, static, deprecated final, transient, volatile, enum Reverse Engineering (Source Code Analysis) © SERG

Example Query • Assume that we want to find all the methods in a specific file (in this case, World. java) that start with ‘get’. Our query would look like the following: bdef sim. xml m "get. *" filename="World. java” World. java is a part of a Discrete Event Simulator that contains information about the simulation environment Reverse Engineering (Source Code Analysis) © SERG

Example Results (bdef) bdef sim. xml m "get. *" filename="World. java" get. World. Array: World: World. java: public: false: false: false: false: get. World. String: World. java: public: false: false: get. World. Mask. String: World. java: public: false: false: get. Empty: World. java: public: false: false: get. Width: World. java: public: false: false: get. Height: World. java: public: false: false: Reverse Engineering (Source Code Analysis) © SERG

Results Explained • The bdef query resulted in a collection of : separated lists. The data in the columns mean the following: – name is the name of the method – class is the class the method belongs too – filename the file containing this method – scope the scope of the method – static if the method is static – deprecated if the method is deprecated – final if the method is final – abstract if the method is abstract – varargs if the method uses variable arguments – bridge if the method is a bridge – native if the method is native – synchronized if the method is synchronized – return the method’s return type – parameters the types of parameters accepted Reverse Engineering (Source Code Analysis) © SERG

Exercise • This exercise uses some Unix utilities along with our use of bdef. The exercise involves two things: – – Counting the number of methods of class World (in World. java). Printing out a list of methods in the form of their name, return type, and parameter list. Reverse Engineering (Source Code Analysis) © SERG

Using Unix (Part One) • In order to count the number of lines of a document, one can use the command line tool wc. – The –l option makes it count lines. – Piping to it makes it count the lines of output from a program. {bdef query} | wc –l counts the number of lines in a bdef query. Reverse Engineering (Source Code Analysis) © SERG

The solution is … • The solution to the first problem is: bdef sim. xml m ". *" filename="World. java" | wc -l Reverse Engineering (Source Code Analysis) © SERG

Using Unix (Part Two) • For the second question, we will again use the unformatted output of bdef. – This time, we’ll take note of the format of the unformatted output! We’ll keep this limited to the case of unformatted output for methods. – Each field of the unformatted output is delimited by a colon. The fields we care about are the name, returntype, and parameter-list fields. These are fields 1, 13, and 14, respectively. Reverse Engineering (Source Code Analysis) © SERG

Using Unix (Part Two) • The final piece in the puzzle of displaying the specific fields is getting the fields themselves out of the output. – The cut utility will do nicely. We can send it a delimiter, and a list of field numbers for a file, and it will return those fields for each line. – The delimiter flag for cut is –d. The field numbers delimiter is –f, followed by a series of comma separated numbers. Reverse Engineering (Source Code Analysis) © SERG

The solution is … • Our target query is thus: bdef sim. xml m ". *" class="World" | cut -d ": " -f 1, 13, 14 Reverse Engineering (Source Code Analysis) © SERG

Output for Exercise • Question One: 13 • Question Two: <init>: : (int, ) remove. Entity: : (Location, ) add. Entity: : (Location, ) check. Bounds: boolean: (Location, ) check. Location: boolean: (Location, ) get. World. Array: char[][]: () get. World. String: java. lang. String: (char[][], ) get. World. String: java. lang. String: () get. World. Mask. String: java. lang. String: (java. util. Vector, ) set. Box: : (char[][], int, char, ) get. Empty: char: () get. Width: int: () get. Height: int: () <clinit>: : () • Not very pretty, but useful (we hope…). Reverse Engineering (Source Code Analysis) © SERG

bref • bref is a tool that displays relationship information by linking one entity to another Reverse Engineering (Source Code Analysis) © SERG

bref Syntax bref xml kind 1 name 1 kind 2 name 2 – kind 1 and kind 2 are entity kinds – name 1 and name 2 are entity names – xml the XML file containing the database Reverse Engineering (Source Code Analysis) © SERG

Example Query • Here’s a query to find all class-class relationships in the database. bref sim. xml c “. *” Reverse Engineering (Source Code Analysis) © SERG

Example Results (bref) • bref sim. xml c “. *” c “. * "Auto. Car" -> "Car” "Auto. Control" -> "java. lang. Object" "Car" -> "Entity” "Car. Control. Exception" -> "java. lang. Exception" "Car. Crash. Exception" -> "java. lang. Exception" "Car. Move. Controller" -> "Entity" "Car. Out. Of. Bounds" -> "java. lang. Exception" "Car. Park. Traffic. Generator" -> "Entity" …………… Reverse Engineering (Source Code Analysis) © SERG

Results Explained • bref returned a list of classes. • Each line represents a relationship between the entities • The entity on the right is the first entity asked for • The entity of the left is the second entity asked for Reverse Engineering (Source Code Analysis) © SERG

Exercise – bref • In these exercises, we’ll examine various relations between the entities of a system. • We’ll go over: – Inheritance relationships. – Method-Method relationships. – How to write a shell script using BAT tools Reverse Engineering (Source Code Analysis) © SERG

Exercise #1 • We’ve already seen how to find the entire inheritance tree from our example, so this exercise should be easy: – Find all the classes that Entity inherits from, and all the classes that subclass it. Reverse Engineering (Source Code Analysis) © SERG

Inheritance Relation • The relation between classes that we are interested in is subclassing. • But which entity in the relation subclasses the other? – The answer is that the first entity subclasses the second. Reverse Engineering (Source Code Analysis) © SERG

Inheritance Relation (Cont’d) – The answer to the question “which class is Entity a subclass of” is: bref sim. xml c “Entity” c “. *” – We can analogously find which classes subclass Entity : bref sim. xml c “. *” c “Entity “ Reverse Engineering (Source Code Analysis) © SERG

Exercise #2 • This exercise concentrates on method-tomethod relations. • Our task is to find what the fan-in and fanout of a function are. • We’ll use World. add. Entity function in the example Reverse Engineering (Source Code Analysis) © SERG

Definition: Fan-In/Fan-Out • Fan-In – The fan-in of a function/method is the number of functions/methods that invoke that method. • Fan-Out – The fan-out of a function/method is the number of functions/methods that it invokes. Reverse Engineering (Source Code Analysis) © SERG

Finding Fan-In, Fan-Out • The fan-in of a method can be calculated thusly: bref sim. xml m ". *" m "World. add. Entity" | wc -l • The fan-out of a method can be calculated analogously: bref sim. xml m "World. add. Entity" m ". *" | wc -l Reverse Engineering (Source Code Analysis) © SERG

Exercise #3 • In this Exercise, we’ll write a shell script to determine if one class is an ancestor or a descendent of another. Reverse Engineering (Source Code Analysis) © SERG

Descendent Relation • A class X is an descendent of class Y if X subclasses Y, or X’s superclass is a descendent of Y. • This sets up a nice recursion, which will make our job easy. Reverse Engineering (Source Code Analysis) © SERG

Shell Scripting • Our first step is to come up with an exact specification of what we want: – Given two classes, D and A, our script should report a 1 if D is an descendent of A, and 0 otherwise. Reverse Engineering (Source Code Analysis) © SERG

Shell Scripting… • Our first coding step is to determine what shell to use. For this exercise, we’ll be using the C shell. • This makes our shebang line like: #!/bin/csh Reverse Engineering (Source Code Analysis) © SERG

Shell Scripting • To make this a little nicer to look at, we’ll make a few small helper-scripts… – One to return whether one class subclasses another. – One to return the ‘name’ field from unformatted BAT output. – One to return the names of all the classes that inherit from a given class. Reverse Engineering (Source Code Analysis) © SERG

Helper Script (does_subclass) • Our first script is pretty simple: #!/bin/csh @ z = `bref $1 c $2 c $3 | wc -l` != 0 echo ${z} Reverse Engineering (Source Code Analysis) © SERG

Helper Script (get_name) • Our get_name script only has to return the value of one field. We’ll just make a small script to do it. cut -d " " -f 1 Reverse Engineering (Source Code Analysis) © SERG

Helper Script (subclasses) • A script to get all the subclasses is also relatively trivial: bref $1 c ". *" c $2 |get_name Reverse Engineering (Source Code Analysis) © SERG

The Actual Script (ancestor) • Since our relation is a recursive one, we have to start our code by taking care of the base case (which is that D is a subclass of A. Parent-Child relationship…). #!/bin/csh if (`bref $1 c $2 c $3 | wc -l ` != 0) then echo 1 exit endif Reverse Engineering (Source Code Analysis) © SERG

The Rest of the Script • The rest of the script deals with the recursion. We have to check every subclass to see if it is an ancestor of the target class. foreach child (`subclasses $1 $3`) if (`ancestor $1 $2 $child`) then echo 1 exit endif end Reverse Engineering (Source Code Analysis) © SERG

However… • There’s a better way to do this, which would be to traverse up from the descendent. – There can be multiple subclasses to any class. – In Java, there is only one superclass to a class. • We’ll call this the ancestor relation, defined as: – X is an ancestor of Y if X is Y’s superclass, – or X is an ancestor of Y’s superclass. • We’ll write two little helper scripts to do the rewrite. Reverse Engineering (Source Code Analysis) © SERG

Helper Scripts, II (other_name) • A script to get the name of the second entity of a relation could be useful. cut -d " " -f 3 Reverse Engineering (Source Code Analysis) © SERG

Helcper Scripts, II (parent) • A second script, to return the parent of a class, if it exists, would be: #!/bin/csh bref $1 c $2 c ". *" | other_name Reverse Engineering (Source Code Analysis) © SERG

Making the Finished Product • First take care of the base case of the recursion: #!/bin/csh if (`other_name $1 $2 $3`) then echo 1 exit endif Reverse Engineering (Source Code Analysis) © SERG

Last Bit o’ Code • The rest of the code deals with recursing up the inheritance tree… if (`parent $1 $2 | wc -l ` != 0) then ancestor $1 `parent $1 $2` $3 else echo 0 endif Reverse Engineering (Source Code Analysis) © SERG

Visualizing Relationships • We will be using DOT and Graphviz to visualize BAT relationships – dot: Used to draw a ‘directed graph. ’ – Graphviz: Visualizes DOT format Reverse Engineering (Source Code Analysis) © SERG

Graphs (Definition) • A graph G(V, E) is a set of vertices, V, and a set of edges, E. • For each edge e in E, there are two vertices, (x, y), in V such that E is an edge between x and y. Reverse Engineering (Source Code Analysis) © SERG

Graph Details • Edge Crossings • Directed Graphs • Parallel Edges Reverse Engineering (Source Code Analysis) © SERG

Graph Examples • A road map of a large area is a graph. Cities are vertices, and roads are edges. • An inheritance tree is a directed graph. • A call tree is a graph. Reverse Engineering (Source Code Analysis) © SERG

DOT Format digraph mdg { "First" -> "java. lang. Object" "First" -> "Second" -> "java. lang. Object" "Second" -> "java. lang. System" "Second" -> "java. io. Print. Stream" } Reverse Engineering (Source Code Analysis) © SERG

Relationship to DOT • The relationship queries already return in DOT format, minus the header. • All we need to do is append the following to the head: – digraph mdg { • And the following to the tail: – } Reverse Engineering (Source Code Analysis) © SERG

XSLT • Both bdef and bref are wrappers around XSLT queries • XSLT/XPATH – Used to query the database. – Firefox can render XSLT stylesheets over XML datasets Reverse Engineering (Source Code Analysis) © SERG

XSLT/XPATH Tutorials/Tools • References – http: //www. w 3 schools. com/ – http: //www. zvon. org/xxl/XSLTreference/Output/index. html – http: //www. xml. com/pub/a/2000/08/holman/index. html • Tools – xsltproc on *nix systems – Windows: http: //www. microsoft. com/downloads/details. aspx? familyid=2 fb 55371 c 94 e-4373 -b 0 e 9 -db 4816552 e 41&displaylang=en – Firefox can apply XSLT stylesheets Reverse Engineering (Source Code Analysis) © SERG

Source Code Analysis Using Chava Reverse Engineering (Source Code Analysis) © SERG

What is Static Analysis? • Mining source code for information. • Using that information to present abstractions of, and answer questions about, software structure. Reverse Engineering (Source Code Analysis) © SERG

What can we get from source code analysis? • Type of information is model dependent – In almost any language, we can find out information about variable usage: Who? Where? etc. – In an OO environment, we can find out which classes use other classes, which are a base of an inheritance structure, etc. – We can also find potential blocks of code that can never be executed in running the program (dead code). Reverse Engineering (Source Code Analysis) © SERG

Chava • Is a tool that lets us perform static analysis on Java programs (source or class files). – Builds a database of entities in a system. – Builds a database of relationships in a system. – Includes several tools for querying the databases for data, and some tools for visualizing results. Reverse Engineering (Source Code Analysis) © SERG

Entities • ‘Entities’ are individuals that live in the system, and attributes associated with them. Some examples: – Classes, along with information about their superclass, their scope, and ‘where’ in the code they exists. – Methods/functions and what their return type or parameter list is, etc. – Variables and what their types are, and whether or not they are static, etc. Reverse Engineering (Source Code Analysis) © SERG

Relationships • ‘Relationships’ are interactions between the entities in the system. Relationships include: – Classes inheriting from one another. – Methods in one class calling the methods of another class, and methods within the same class calling one another. – One variable referencing another variable. Reverse Engineering (Source Code Analysis) © SERG

Creating Chava Databases • Chava takes java/class files, and turns them into data files (. A ext) that can be integrated into a database – Create a. A file for a given Java file: chava –c filename. java – Create. A files for all Java files in directory: chava –c *. class Reverse Engineering (Source Code Analysis) © SERG

Pulling it all together… • Chava then takes. A files and creates the databases. – Create databases out of two. A files: chava –l f 1. A f 2. A – Create databases for all. A files in directory: chava –l *. A Reverse Engineering (Source Code Analysis) © SERG

Chava Tools • cdef/vdef – Used to query the entities database. • cref/vref – Used to query the relationship database. • dagger/dot – A visualization tool. Takes information from chava databases and displays it as a graph. Reverse Engineering (Source Code Analysis) © SERG

cdef/vdef • cdef takes information from the entities databased on a query, and returns the results in an ascii-table. • vdef actually shows the code of the entities from a query. Reverse Engineering (Source Code Analysis) © SERG

Syntax • cdef and vdef share the same syntax: {vdef|cdef} entity_kind entity_name [attr=val]. . – entity_kind is the ‘type’ of entity to retrieve. – entity_name is a pattern to match for names of entities. – attr=val are bindings to match for attributes of the entity Reverse Engineering (Source Code Analysis) © SERG

Entity Kinds • Chava recognizes several types of entity ‘kinds’ for use in the cdef/vdef/cref/vref commands. • • p f m c l s i - is for Package is for File is for Method is for Class is for Field is for String is for Interface is a match for any entity_kind Reverse Engineering (Source Code Analysis) © SERG

Entity Names • An entity name can assume many forms – Explicit name (e. g. , ‘my. Temp. String. Var’) – Wild-card Pattern (e. g. , ‘my. Temp*’) – A complete wild-card, denoted with ‘-’ Reverse Engineering (Source Code Analysis) © SERG

Attribute=Value settings are used to further restrict a query based on some condition. • The most common restriction is to restrict to a specific file, or to filter out a file. E. g. , • Attribute=Value cdef - - file=File. IDo. Like. java cdef - - file!=File. IDo. NOTLike. java Reverse Engineering (Source Code Analysis) © SERG

Example Query • Assume that we want to find all the methods in a specific file (in this case, ANSIDisplay. java) that start with ‘get’. Our query would look like the following: cdef m ‘get*’ file=. /ANSIDisplay. java • Or, to see the code… vdef m ‘get*’ file=. /ANSIDisplay. java Reverse Engineering (Source Code Analysis) © SERG

Example Results (cdef) • cdef m ‘get*’ file=ANSIDisplay. java name =========== String get. Escape. Sequen scope ======= public public file ========= ANSIDisplay. java bline ===== 76 38 93 118 139 eline ===== 82 42 108 128 153 Reverse Engineering (Source Code Analysis) © SERG

Results Explained • The cdef query resulted in a table with several columns. The data in the columns mean the following: – name: The name of the entity. – scope: The scope of the entity within its ‘parent’ entity (the entity it resides in). – file: The name of the file that the entity is in. – bline: The line that the entity begins on. – eline: The line that the entity ends on. Reverse Engineering (Source Code Analysis) © SERG

Example Results (vdef) • vdef m ‘get*’ file=ANSIDisplay. java (partial results) public static String get. Escape. Sequence(int colour, boolean foreground) { colour = set. Colour(colour, foreground); return (ESCAPE + Integer. to. String(colour) + "m"); } public static String get. Escape. Sequence(int value) { if (!ANSIDisplay. Switch. Check. valid. Switch(value)) throw new Illegal. Argument. Exception("Bad Switch"); return (ESCAPE + Integer. to. String(value) + "m"); } Reverse Engineering (Source Code Analysis) © SERG

Results Explained • vdef printed out the entities we asked about, exactly how they appear in the source code. Reverse Engineering (Source Code Analysis) © SERG

Finding all File Names • Knowing all the file names could be important, so let’s see how to do that with chava. – We want to use cdef for this, and just have chava output a list of file names. – We also want to restrict the entity_kind to that of file. If you remember, ‘f’ is the type for file. – We also want any file in the database to be listed, so we want to match against any entity_name. ‘-’ will do. cdef f - Reverse Engineering (Source Code Analysis) © SERG

Exercise • It would be nice to know how a class interacts with its superclass. • We’ll take a peek at this with the classes ANSIColour. Printer and ANSIPrinter. Reverse Engineering (Source Code Analysis) © SERG

Class-Superclass • This problem is a bit more than just one cdef/vdef command. First step… – We need to see how ANSIColour. Printer calls its super-constructor. – We want to see the calls, so we’ll use vdef. – Constructors are methods in chava. Reverse Engineering (Source Code Analysis) © SERG

Class-Superclass (Step One) • The query we need to see the constructors of ANSIColour. Printer is: vdef m ANSIColour. Printer This results in… Reverse Engineering (Source Code Analysis) © SERG

Class-Superclass Interaction (Step 1 – Results) public ANSIColour. Printer(Output. Stream out) { this(out, m_default. Colour); } public ANSIColour. Printer(Output. Stream out, boolean do. Reset) { this(out, m_default. Colour, do. Reset); } public ANSIColour. Printer(Output. Stream out, ANSICharacter. Colour colour ) { this(out, colour, m_default. Reset); } public. ANSIColour. Printer(Output. Stream out, ANSICharacter. Colour colour , boolean do. Reset) { super(colour, out, do. Reset); } Reverse Engineering (Source Code Analysis) © SERG

Class-Superclass Interaction (Step One – Analysis) • We now know that ANSIColour. Printer accepts: – An Output. Stream, – An ANSICharacter. Colour – A boolean. • When not supplied with either of the last two parameters, the constructor uses some defaults. Reverse Engineering (Source Code Analysis) © SERG

Class-Superclass Interaction (Step Two) • The next step is to examine what ANSIPrinter does in its constructor. • This is basically the same thing as peeking at the ANSIColour. Printer constructors. vdef m ANSIPrinter Reverse Engineering (Source Code Analysis) © SERG

Class-Superclass Interaction (Step Two – Results) public ANSIPrinter(Output. Stream out, ANSIEscape. Sequence. Type sequence) { this(out, sequence, m_default. Reset); } public ANSIPrinter(Output. Stream out, ANSIEscape. Sequence. Type sequence, boolean do. Reset) { this(sequence, out, do. Reset, true); } public ANSIPrinter(Output. Stream out, ANSIEscape. Sequence. Type sequence, boolean do. Reset, boolean reset. On. Leave) { super(out); m_escape = sequence; m_reset = do. Reset; m_reset. On. Finalize = reset. On. Leave; m_show. Escape = false; } Reverse Engineering (Source Code Analysis) © SERG

Class-Superclass Interaction (Step Two – Analysis) • Apparently, the constructor for ANSIPrinter accepts values for: – an Output. Stream – an ANSIEscape. Sequence. Type – two booleans. • From what we see the constructor of ANSIPrinter doing, we know that the constructor does nothing more than just set some variables to what we pass to it. Nothing really that special. Reverse Engineering (Source Code Analysis) © SERG

Another Exercise • This exercise uses some Unix utilities along with our use of cdef/vdef. The exercise involves two things: 1. Counting the number of methods of ANSICharacter. Colour (in ANSICharacter. Colour. java). 2. Printing out a list of methods in the form of their name, return type, and parameter list. Reverse Engineering (Source Code Analysis) © SERG

Using Unix (Part One) • In order to count the number of lines of a document, one can use the command line tool wc. – The –l option makes it count lines. – Piping to it makes it count the lines of output from a program. {cdef query} | wc –l counts the number of lines in a cdef query. Reverse Engineering (Source Code Analysis) © SERG

Using Unix (Part One) • Problem with using wc – wc counts all lines, including the ones for our formatted output table. – Passing the –u option to cdef gives unformatted output, which is very useful for integrating chava with unix tools. The syntax is: cdef [-u] kind name [attr=val] Reverse Engineering (Source Code Analysis) © SERG

The solution is … • The solution to the first problem is: cdef –u m – file=. /ANSICharacter. Colour. java | wc -l Reverse Engineering (Source Code Analysis) © SERG

Using Unix (Part Two) • For the second question, we will again use the unformatted output of cdef. – This time, we’ll take note of the format of the unformatted output! We’ll keep this limited to the case of unformatted output for methods. – Each field of the unformatted output is delimited by a semicolon. The fields we care about are the name, return-type, and parameter-list fields. These are fields 2, 5, and 9, respectively. Reverse Engineering (Source Code Analysis) © SERG

Using Unix (Part Two) • The final piece in the puzzle of displaying the specific fields is getting the fields themselves out of the output. – The cut utility will do nicely. We can send it a delimiter, and a list of field numbers for a file, and it will return those fields for each line. – The delimiter flag for cut is –d. The field numbers delimiter is –f, followed by a series of comma separated numbers. Reverse Engineering (Source Code Analysis) © SERG

The solution is … • Our target query is thus: cdef –u m – file=. /ANSICharacter. Colour. java | cut –d’; ’ –f 2, 5, 9 Reverse Engineering (Source Code Analysis) © SERG

Output for Exercise • Question One: 13 • Question Two: ANSICharacter. Colour; void; (acin. common. ansi. ANSIColour, acin. common. ansi. ANSIColour) create; acin. common. ansi. ANSICharacter. Colour; (acin. common. ansi. ANSIColour, int) create; acin. common. ansi. ANSICharacter. Colour; (acin. common. ansi. ANSIColour, java. lang. String) create; acin. common. ansi. ANSICharacter. Colour; (int, acin. common. ansi. ANSIColour) create; acin. common. ansi. ANSICharacter. Colour; (int, int) create; acin. common. ansi. ANSICharacter. Colour; (int, java. lang. String) create; acin. common. ansi. ANSICharacter. Colour; (java. lang. String, acin. common. ansi. ANSIColour) create; acin. common. ansi. ANSICharacter. Colour; (java. lang. String, int) create; acin. common. ansi. ANSICharacter. Colour; (java. lang. String, java. lang. String) get. ANSIString; java. lang. String; () get. Background; acin. common. ansi. ANSIColour; () get. Foreground; acin. common. ansi. ANSIColour; () • Not very pretty, but useful (we hope…). Reverse Engineering (Source Code Analysis) © SERG

cref/vref • cref is a tool that displays information from the Chava relationship database, returning the results in a table. • vref displays the actual entities involved in a relationship. Reverse Engineering (Source Code Analysis) © SERG

Syntax • cref and vref share the same syntax {cref|vref} kind 1 name 1 kind 2 name 2 [attr=val]. – kind 1 and kind 2 are entity kinds – name 1 and name 2 are entity names – Attributes are a bit different… Reverse Engineering (Source Code Analysis) © SERG

cref/vref Attributes • Attr=val pairs in cref/vref are different because they have to deal with two different entities. This is solved by appending a ‘ 1’ or a ‘ 2’ on the attribute. E. g. , – file 1=my. File. java – file 2!=your. File. java Reverse Engineering (Source Code Analysis) © SERG

Example Query • Here’s a query to find all class-class relationships in the database. cref c – or, to see the results: vref c – Reverse Engineering (Source Code Analysis) © SERG

Example Results (cref) • cref c – kind 1 ===== class class class class name 1 ==== ANSIChar ANSIColo ANSICurs ANSIDisp ANSIEsca ANSIPrin file 1 ======== ANSICharacter. Col ANSIColour. java ANSIColour. Printe ANSICursor. Move. j ANSICursor. Move. Se ANSIDisplay. java ANSIDisplay. Switc ANSIEscape. Sequen ANSIPrinter. java ANSIPrinter. Map. j ANSIPrinter. Optio kind 2 ===== class class class class name 2 file 2 ============ ANSIEscape. Sequen ANSIPrinter. java ANSIPrinter. Map. j Object ANSIEscape. Sequen Object Print. Stream Object rk == su su su su Reverse Engineering (Source Code Analysis) © SERG

Results Explained • cref returned a table. The columns are just like cdef columns, except some have a ‘ 1’ and some have a ‘ 2’ appended. • Columns with a ‘ 1’ appended refer to the first entity. • Columns with a ‘ 2’ refer to the second entity. Reverse Engineering (Source Code Analysis) © SERG

That last column… • The last column, rk, denotes the kind of relationship. Its values can be: • • • Reference Fieldread Fieldwrite Implements Subclass Reverse Engineering (Source Code Analysis) © SERG

Example Results (vref) • vref c – (partial results) RECORD NUMBER 0 ### ANSICharacter. Colour. java ### public class ANSICharacter. Colour extends ANSIEscape. Sequence. Type { private ANSIColour m_foreground; private ANSIColour m_background; /** * Method to create an <code>ANSICharacter. Colour</code> from two integers representing the foreground and background colour, as defined in <code>ANSIColour. Constants</code>. * * @param foreground The value representing the colour to be the foreground. * @param background The value representing the colour to be the background. * @exception java. lang. Illegal. Argument. Exception Thrown if the foreground and background values aren't valid ANSI colours. */ public static ANSICharacter. Colour create(int foreground, int background) { return new ANSICharacter. Colour(new ANSIColour(foreground, true), new ANSIColour(background, false)); } Reverse Engineering (Source Code Analysis) © SERG

Exercise – cref/vref • In these exercises, we’ll examine various relations between the entities of a system. • We’ll go over: – Inheritance relationships. – Method-Method relationships. – How to write a shell script using Chava tools Reverse Engineering (Source Code Analysis) © SERG

Exercise #1 • We’ve already seen how to find the entire inheritance tree from our example, so this exercise should be easy: – Find all the classes that ANSIEscape. Sequence. Type inherits from, and all the classes that subclass it. Reverse Engineering (Source Code Analysis) © SERG

Inheritance Relation • The relation between classes that we are interested in is subclassing. • But which entity in the relation subclasses the other? – The answer is that the first entity subclasses the second. Reverse Engineering (Source Code Analysis) © SERG

Inheritance Relation (Cont’d) – The answer to the question “which class is ANSIEscape. Sequence. Type a subclass of” is: cref c ANSIEscape. Sequence. Type c - – We can analogously find which classes subclass ANSIEscape. Sequence. Type: cref c - c ANSIEscape. Sequence. Type Reverse Engineering (Source Code Analysis) © SERG

Exercise #2 • This exercise concentrates on method-tomethod relations. • Our task is to find what the fan-in and fanout of a function are. Reverse Engineering (Source Code Analysis) © SERG

Definition: Fan-In/Fan-Out • Fan-In – The fan-in of a function/method is the number of functions/methods that invoke that method. • Fan-Out – The fan-out of a function/method is the number of functions/methods that it invokes. Reverse Engineering (Source Code Analysis) © SERG

Finding Fan-In, Fan-Out • A key piece of information to know here is that the –u option from cdef works in cref. • The fan-in of a method can be calculated thusly: cref –u m – m my_method | wc –l • The fan-out of a method can be calculated analogously: cref –u m my_method m - | wc –l Reverse Engineering (Source Code Analysis) © SERG

Exercise #3 • In this Exercise, we’ll write a shell script to determine if one class is an ancestor or a descendent of another. Reverse Engineering (Source Code Analysis) © SERG

Descendent Relation • A class X is an descendent of class Y if X subclasses Y, or X’s superclass is a descendent of Y. • This sets up a nice recursion, which will make our job easy. Reverse Engineering (Source Code Analysis) © SERG

Shell Scripting • Our first step is to come up with an exact specification of what we want: – Given two classes, D and A, our script should report a 1 if D is an descendent of A, and 0 otherwise. Reverse Engineering (Source Code Analysis) © SERG

Shell Scripting… • Our first coding step is to determine what shell to use. For this exercise, we’ll be using the C shell. • This makes our shebang line like: #!/bin/csh Reverse Engineering (Source Code Analysis) © SERG

Shell Scripting • To make this a little nicer to look at, we’ll make a few small helper-scripts… – One to return whether one class subclasses another. – One to return the ‘name’ field from unformatted chava output. – One to return the names of all the classes that inherit from a given class. Reverse Engineering (Source Code Analysis) © SERG

Helper Script (does_subclass) • Our first script is pretty simple: #!/bin/csh @ z = `cref –u c $1 c $2 | wc –l` != 0 echo ${z} Reverse Engineering (Source Code Analysis) © SERG

Helper Script (get_name) • Our get_name script only has to return the value of one field. We’ll just make a small awk script to do it. awk –F ‘; ’ ‘{print $3}’ Reverse Engineering (Source Code Analysis) © SERG

Helper Script (subclasses) • A script to get all the subclasses is also relatively trivial: cref –u c – c $1 | get_name Reverse Engineering (Source Code Analysis) © SERG

The Actual Script (ancestor) • Since our relation is a recursive one, we have to start our code by taking care of the base case (which is that D is a subclass of A. Parent-Child relationship…). if (`cref –u c $1 c $2 | wc -l` != 0) then echo 1 exit endif Reverse Engineering (Source Code Analysis) © SERG

The Rest of the Script • The rest of the script deals with the recursion. We have to check every subclass to see if it is an ancestor of the target class. foreach child (`subclasses $2`) if (`ancestor $1 $child`) then echo 1 exit endif end Reverse Engineering (Source Code Analysis) © SERG

However… • There’s a better way to do this, which would be to traverse up from the descendent. – There can be multiple subclasses to any class. – In Java, there is only one superclass to a class. • We’ll call this the ancestor relation, defined as: – X is an ancestor of Y if X is Y’s superclass, – or X is an ancestor of Y’s superclass. • We’ll write two little helper scripts to do the rewrite. Reverse Engineering (Source Code Analysis) © SERG

Helper Scripts, II (other_name) • A script to get the name of the second entity of a relation could be useful. awk -F '; ' '{print $17}' Reverse Engineering (Source Code Analysis) © SERG

Helper Scripts, II (parent) • A second script, to return the parent of a class, if it exists, would be: #!/bin/csh cref -u c $1 c - | other_name Reverse Engineering (Source Code Analysis) © SERG

Making the Finished Product • First take care of the base case of the recursion: if (`does_subclass $1 $2`) then echo 1 exit endif Reverse Engineering (Source Code Analysis) © SERG

Last Bit o’ Code • The rest of the code deals with recursing up the inheritance tree… if (`parent $1 | wc –l` != 0) then ancestor `parent $1` $2 else echo 0 endif Reverse Engineering (Source Code Analysis) © SERG

Visualizing Chava • There are two tools we’ll be using to visualize chava queries. – dagger: Lets us use a cref-esque query to create a ‘directed graph. ’ – dot: Used to draw a ‘directed graph. ’ Reverse Engineering (Source Code Analysis) © SERG

Graphs (Definition) • A graph G(V, E) is a set of vertices, V, and a set of edges, E. • For each edge e in E, there are two vertices, (x, y), in V such that E is an edge between x and y. Reverse Engineering (Source Code Analysis) © SERG

Graph Details • Edge Crossings • Directed Graphs • Parallel Edges Reverse Engineering (Source Code Analysis) © SERG

Graph Examples • A road map of a large area is a graph. Cities are vertices, and roads are edges. • An inheritance tree is a directed graph. • A call tree is a graph. Reverse Engineering (Source Code Analysis) © SERG

The dagger Tool • The dagger tool takes a cref-style query, and returns the results as a graph of the relationships. Reverse Engineering (Source Code Analysis) © SERG

Syntax • dagger syntax is exactly like cref syntax (except for lack of options). dagger kind 1 name 1 kind 2 name 2 Reverse Engineering (Source Code Analysis) © SERG

dagger to dot • dagger only creates a representation describing a graph. • dot takes that representation and outputs something that can be visualized. – Can make dotty files. – Can also make postscript files. dagger kind 1 name 1 kind 2 name 2 | dot -Tps Reverse Engineering (Source Code Analysis) © SERG

Example Query • A sample query will show just how the output of dagger -> dot looks. • A good thing to check is the class inheritance heirarchy. – We already know the cref query for this. – The dagger query is dagger c – | dot –Tps > classes. ps Reverse Engineering (Source Code Analysis) © SERG

Viewing Post. Script • A good Post. Script viewer is ghostview. – The command to use ghostview is ggv <file> • Use ghostview to look at the class heirarchy graph that you just created. Reverse Engineering (Source Code Analysis) © SERG

Does Chava have siblings? • Chava is really a tool that uses the CIA system, from AT&T Labs - Research. • The CIA system can be extended to any type of structured language. • Other implementations exist for: – C/C++, HTML, ksh, etc. Reverse Engineering (Source Code Analysis) © SERG