PRACTICAL ANALYSIS OF NONTERMINATION IN LARGE LOGIC PROGRAMS

PRACTICAL ANALYSIS OF NONTERMINATION IN LARGE LOGIC PROGRAMS Senlin Liang and Michael Kifer

MOTIVATION High-level LP languages, e. g. , � SILK http: //silk. semwebcentral. org � Flora-2 http: //flora. sourceforge. net are designed to be suitable for knowledge engineers, who are not programmers KBs created by engineers typically � are complex, large � stress the capabilities of the underlying engine => Non-termination happens often – very hard to debug To address this, we developed � Terminyzer – a non-Termination analyzer (this extends our previous work in PADL-13)

OUTLINE Preliminaries: causes of non-termination, tabling, and forest logging Adding ids to rules Terminyzer: analyses causes, repairs problems Experiments Conclusion and future work

CAUSES OF NON-TERMINATION Cause 1: loops in SLD-resolution Example p(X) : - p(X). ? - p(a). Solution: tabling [SW 12] � Caches calls to subgoals, which cuts recursive loops If enough predicates are tabled then � Each subgoal is tabled once � Each answer is tabled once � Evaluation terminates if there are finitely many subgoals and answers [SW 12] Terry Swift and David S. Warren. XSB: Extending prolog with tabled logic programming. TPLP’ 12.

CAUSES OF NON-TERMINATION (CONT’D) Cause 2: � Engine supports tabling � But the program generates infinitely many tabled subgoals Example p(X) : - p(f(X)). ? - p(a). Subgoals to be tabled: p(a), p(f(a)), p(f(f(a))), . . . Solution: subgoal abstraction [RS] � Abstracts subgoals that are deeper than a threshold Assuming threshold = 2, p(f(f(f(a)))) would be abstracted to p(f(f(X))), X = f(a) � Guarantees that there will be only a finite number of tabled subgoals [RS] F. Riguzzi and T. Swift. Terminating evaluation of logic programs with finite three-valued models. ACM on Computational Logic. To appear.

CAUSES OF NON-TERMINATION (CONT’D) Cause 3: Engine supports both tabling and subgoal abstraction � But infinitely many answers � Example p(a). p(f(X)) : - p(X). ? - p(X). Answers to be derived: p(a), p(f(a)), p(f(f(a))), … Solution: does not exist Halting problem is undecidable � Whether a program has a finite number of answers is undecidable � We can only try to help the user to deal with the issue � If user really intended the program to be the way it is � We need a way to limit output. E. g. , bounded rationality [BS 13] Our focus: Unexpected non-termination (ie, when it’s a bug) [BS 13] B. Grosof and T. Swift. Radial restraint: A semantically clean approach to bounded rationality for logic programs. AAAI’ 13

TABLING AND FOREST LOGGING Tabling needs no introduction Forest logging is a new tracing facility in XSB Events Logs Calls to tabled subgoals E. g. parent calls child tabled_call(child, parent, status, timestamp) neg_tabled_call(child, parent, status, timestamp) Answer derivations E. g. ansr is derived for sub new_answer(ansr, sub, timestamp) new_delayed_answer(ansr, sub, delayed_lits, timestamp) Return answers to consumers E. g. ansr for child is retuned to parent answer_return(ansr, child, parent, timestamp) delayed_answer_return(ansr, child, parent, timestamp) Subgoal completions E. g. sub is completed(sub, scc_num, timestamp) completed(sub, early_completed, timestamp) Other events Irrelevant to our discussion where Timestamp preserves the order of events � status = new, complete, incomplete �

ADDING IDS TO RULES – KEY INSIGHT Add unique ids to rules s. t. tabled subgoals remember their host rules: Each tabling declaration : - table p/n is changed to : - table p/(n+1) For a query ? - p(x 1, …, xn) � If p/n is tabled, then Change it to ? - p(x 1, …, xn, Newvar) Chop off the last arguments of returned answers � Otherwise, the query stays the same

TERMINYZER OVERVIEW Two versions Version 1 (most precise) requires: � Tabling � Forest logging � Subgoal abstraction Version 2 requires only � Tabling � Forest logging Currently only XSB has all three features, but: � Several systems have tabling � Forest logging info exists internally in all of them – just needs to be exposed to the user => at least Version 2 is easily portable

TERMINYZER OVERVIEW Suppose a query does not terminate, Terminyzer then � Analyzes an execution forest log � Determines the causes of non-termination The exact sequence of unfinished tabled subgoals, and the host rule id of each subgoal Subgoals forming recursive cycles � Rectifies some causes of misbehavior (heuristically)

CALL SEQUENCE ANALYSIS Identifies the exact sequence of unfinished calls and their host rule ids that lead to a non-termination � Find unfinished subgoals – whose answers have not been completely derived – by unfinished(Child, Parent, Timestamp) : tabled_call(Child, Parent, new, Timestamp), not_exists(completed(Child, SCCNum, Timestamp 1)). not_exists is the XSB well-founded negation operator; existentially quantifies SCCNum and Timestamp 1. unfinished(child, parent, timestamp) says that � Sort Subgoal parent calls subgoal child, and it happened at timestamp Neither child nor parent have been completely evaluated unfinished calls by their timestamps � Host rule ids are kept in the last arguments of childsubgoals

CALL SEQUENCE ANALYSIS (CONT’D) Example 1 @!r 1 p(a). @!r 5 r(X) : - r(X). @!r 2 p(f(X)) : - q(X). @!r 6 r(X) : - p(X), s(X). @!r 3 q(b). @!r 7 s(f(b)). @!r 4 q(g(X)) : - p(X). ? - r(X). where @!ruleid is the syntax to assign rule ids Its unfinished calls – the red ones form a nonterminating loop unfinished(r(_h 9900, _h 9908), root, 0) – root is for intial query unfinished(r(_h 9870, r 5), r(_h 9870, _h 9889), 8) unfinished(r(_h 9840, r 5), 11) unfinished(p(_h 9810, r 6), r(_h 9810, r 5), 12) unfinished(q(_h 9780, r 2), p(_h 9780, r 6), 16) unfinished(p(_h 9750, r 4), q(_h 9750, r 2), 20) unfinished(q(_h 9720, r 2), p(_h 9720, r 4), 24) Next, we will find recursive cycles

CALL SEQUENCE ANALYSIS (CONT’D) Unfinished calls can be represented using an unfinished-call graph UCG = (N, E) � � N: the set of unfinished subgoals E: {(parent, child) |unfinished(child, parent, ts) is true} For the above example unfinished(r(_h 9900, _h 9908), root, 0) unfinished(r(_h 9870, r 5), r(_h 9870, _h 9889), 8) unfinished(r(_h 9840, r 5), 11) unfinished(p(_h 9810, r 6), r(_h 9810, r 5), 12) Can be represented as unfinished(q(_h 9780, r 2), p(_h 9780, r 6), 16) unfinished(p(_h 9750, r 4), q(_h 9750, r 2), 20) unfinished(q(_h 9720, r 2), p(_h 9720, r 4), 24) where: Each node is represented by the timestamp when it is first called � -1 represents root � Edges are labeled with timestamps of calls � Loops in UCG represent recursive cycles However, not all cycles are causing non-termination � E. g. [8, 8] thousands of subgoals

CALL SEQUENCE ANALYSIS (CONT’D) Assume all predicates are tabled + subgoal abstraction Theorem (Soundness of the call sequence analysis) If there are unfinished calls in a query’s complete trace, then � Call sequence analysis finds the exact sequence of unfinished calls that caused non-termination, and � The ids of the rules that issued these calls Theorem (Completeness of the call sequence analysis) If the evaluation of a query does not terminate, then � There is at least one loop in the UCG for its complete trace, and the loop’s subgoals are responsible for generating infinite number of answers, and � The last argument of each of these subgoals specifies the rule ids from whose bodies these subgoals were called. The complete trace is infinite due to non-termination so, practically speaking We work with only a prefix of the trace by limiting term depth/size or execution time/space � It may produce false negatives, but they are also useful for identifying computational bottlenecks. �

ANSWER FLOW ANALYSIS Recall that not all cycles in UCG are causing nontermination, so we need to refine call sequence analysis Answer flow analysis does precisely that: it identifies the cycles that actually cause nontermination Non-termination happens if and only if a subset of subgoals keeps: � Receiving answers from producers, � Deriving new answers, and � Returning answers to callers Answer flow analysis looks for repeated patterns of answer returns

ANSWER FLOW ANALYSIS (CONT’D) Compute answer-flow patterns (AFP) Answer-return sequence (ARS): the sequence of (child, parent) pairs where child returns answers to parent � Candidate AFP: a sequence cafp s. t. cafp 2+ is a suffix of ARS � AFP: the shortest candidate AFP cafp s. t. its repetition forms the maximal suffix of ARS among all candidate AFP’s � In previous Example 1 ARS = [ (p(_h 599, r 4), q(_h 599, r 2)), (q(_h 619, r 2), p(_h 619, r 4)), (p(_h 639, r 4), q(_h 639, r 2)), (q(_h 659, r 2), p(_h 659, r 4)), (p(_h 679, r 4), q(_h 679, r 2)), (q(_h 699, r 2), p(_h 699, r 4)), (p(_h 719, r 4), q(_h 719, r 2)), (q(_h 739, r 2), p(_h 739, r 4)), (p(_h 759, r 4), q(_h 759, r 2)), (q(_h 779, r 2), p(_h 779, r 4))]. where (child, parent) indicates child returns answers to parent � Candidate AFPs are: � � cafp 1 = [(p(_h 759, r 4), q(_h 759, r 2)), (q(_h 779, r 2), p(_h 779, r 4))] cafp 2 = cafp 1 • cafp 1 AFP is cafp 1 AFP captures information flow pattern without redundancy @!r 2 p(f(X)) : - q(X). @!r 4 q(g(X)) : - p(X).

ANSWER FLOW ANALYSIS (CONT’D) An AFP can be represented as an answer-flow graph AFG = (N, E), where the set of subgoals in afp � E: {(child, parent) | (child, parent) ∈ afp} � N: Loops in AFG represent cycles that cause nontermination

ANSWER FLOW ANALYSIS (CONT’D) As before, we assume all predicates are tabled + subgoal abstraction Theorem (Soundness of the answer flow analysis) If the complete trace of a query has an AFP then the query does not terminate. Theorem (Completeness of the answer flow analysis) If the query evaluation does not terminate, then: � There is an AFP in its complete trace, � AFG = (N, E) contains at least one loop, � Every sub ∈ N appears in at least one loop, and � Each edge (sub 1, sub 2)∈E, where sub 1=pred(. . . , ruleid), tells us that sub 2 calls sub 1 from the body of a rule whose id is ruleid.

MORE ON UCG AND AFG Theorem (Relationship between UCG and AFG) Consider the UCG and AFG for a nonterminating forest log, we have: nodes(AFG) ⊂ nodes(UCG) v edges(AFG) ⊂ reverse-edges(UCG) v loops(AFG) ⊆ loops(UCG) v Theorem (No false results for finite traces) If the evaluation of a query, Q, terminates, then both the UCG and the AFG for Q’s trace are empty.

AUTO-REPAIR OF RULES A query does not terminate if � It has infinitely many answers, or � It has a finite number of answers, but one of its subqueries has an infinite number of them In this case: a different evaluation order may terminate the query This case is targeted by our auto-repair heuristic For each unfinished(child, parent, timestamp) � We know the host rule for this call, and the common set of the unbound arguments of parent and child – the arguments whose bindings are to be derived � Thus, to reduce the possibility that parent receives infinite number of bindings from child, one can delay issuing a child-call from its host rule until these arguments are bound

AUTO-REPAIR OF RULES (CONT’D) Example @!r 1 @!r 2 @!r 3 @!r 4 p(a). p(f(X)) : - q(X). q(b). q(g(X)) : - p(X). @!r 5 r(X) : - r(X). @!r 6 r(X) : - p(X), s(X). @!r 7 s(f(b)). ? - r(X). Its unfinished calls are: unfinished(r(_h 9900, _h 9908), root, 0) unfinished(r(_h 9870, r 5), r(_h 9870, _h 9889), 8) unfinished(r(_h 9840, r 5), 11) unfinished(p(_h 9810, r 6), r(_h 9810, r 5), 12) unfinished(q(_h 9780, r 2), p(_h 9780, r 6), 16) unfinished(p(_h 9750, r 4), q(_h 9750, r 2), 20) unfinished(q(_h 9720, r 2), p(_h 9720, r 4), 24) Applying auto-repair @!r 1 @!r 2 @!r 3 @!r 4 p(a). p(f(X)) : - wish(ground(X))^q(X). q(b). q(g(X)) : - wish(ground(X))^p(X). @!r 5 r(X) : - wish(ground(X))^r(X). @!r 6 r(X) : - wish(ground(X))^p(X), s(X). @!r 7 s(f(b)). ? - wish(ground(X))^r(X). Then the query will terminate with X = f(b)

TABLED ENGINES WITHOUT SUBGOAL ABSTRACTION Additional cause of non-termination: infinite number of subgoals Steps � Compute the sequence of unfinished subgoals � Compute simplified subgoal sequence (SSS) out of unfinished subgoal sequence Each unfinished subgoal, predicate(…, ruleid), is simplified to predicate(ruleid) � Find the SSS pattern, as in the case of answer flow pattern � SSS pattern contains the predicates and their rule ids that recursively call one another to form increasingly deep subgoals

TABLED ENGINES WITHOUT SUBGOAL ABSTRACTION (CONT’D) Example @!r 1 p(a). @!r 2 p(X) : - q(f 1(X)). @!r 3 q(X) : - p(f 2(X)). ? - r(a). @!r 4 r(X) : - r(X). @!r 5 r(X) : - p(X), s(X). @!r 6 s(a). Its unfinished calls are: unfinished(r(a, _h 46), root, 0). unfinished(r(a, r 4), r(a, _h 27), 8). unfinished(r(a, r 4), 11). unfinished(p(a, r 5), r(a, r 4), 12). unfinished(q(f 1(a), r 2), p(a, r 5), 16). unfinished(p(f 2(f 1(a)), r 3), q(f 1(a), r 2), 19). unfinished(q(f 1(f 2(f 1(a))), r 2), p(f 2(f 1(a)), r 3), 22). unfinished(p(f 2(f 1(a)))), r 3), q(f 1(f 2(f 1(a))), r 2), 25). unfinished(q(f 1(f 2(f 1(a))))), r 2), p(f 2(f 1(a)))), r 3), 28). unfinished(p(f 2(f 1(f 2(f 1(a)))))), r 3), q(f 1(f 2(f 1(a))))), r 2), 31). unfinished(q(f 1(f 2(f 1(a)))))), r 2), p(f 2(f 1(f 2(f 1(a)))))), r 3), 34). …… SSS = [root, r(_), r(r 4), p(r 5), q(r 2), p(r 3), q(r 2)] SSS pattern = [p(r 3), q(r 2)] – says that it is predicate p of r 3 and predicate q of r 2 that recursively call each other, thus forming increasingly deep nested subgoals

STATUS Unfinished-call/answer flow implemented in SILK and Flora-2 SILK has a GUI, Flora-2’s underway Rule Ids are crucial for practicality Auto-repair: not implemented yet

EXPERIMENTS System � Dual core 2. 4 GHz Lenovo X 200 with 3 GB RAM � Ubuntu 11. 04 with Linux kernel 2. 6. 38 Small programs � They took a tiny fraction of a second to analyze � Correctness of analyses is manually verified Large programs: one biology ontology from SILK � KB size: Flora-2 program with 4, 774 rules and 919 facts Compiled into XSB’s 5, 500+ rules and 1, 000+ facts � Logs produced until evaluation consumed all memory Size: ~2 GB Number of records: ~14 M � Took 170 seconds

CONCLUSIONS Terminyzer – a tool for analyzing non-termination Future work: � Implement auto-repair � better auto-repair algorithms Comparison with others: � All other work deals with underpowered logic engines that are so last Century (Prolog) � Or with trying to find sufficient conditions for termination (different focus)

Thank you!

FOREST LOGGING – EXAMPLE : - table path/2. edge(1, 2). edge(1, 3). path(X, Y) : - edge(X, Y). ? - path(1, Y). edge(2, 1). path(X, Y) : - edge(X, Z), path(Z, Y).