Testing Atomicity of Composed Concurrent Operations Ohad Shacham

Testing Atomicity of Composed Concurrent Operations Ohad Shacham Nathan Bronson Alex Aiken Mooly Sagiv Martin Vechev Eran Yahav Tel Aviv University Stanford University Tel Aviv University ETH & IBM Research Technion

Concurrent Data Structures • Writing highly concurrent data structures is complicated • Modern programming languages provide efficient concurrent collections with atomic semantics … … … …… . .

Challenge Testing the atomicity of composed operations … … … ……

TOMCAT Motivating Example TOMCAT 6. * 5. * attr = new Concurrent. Hash. Map(); … Attribute remove. Attribute(String name){ Attribute val = null; synchronized(attr) /* synchronized(attr) { { */ found = attr. contains. Key(name) ; if (found) { val = attr. get(name); attr. remove(name); } }/* } */ return val; } Invariant: remove. Attribute(name) returns the value it removes from attr or null

remove. Attribute(“A”) { Attribute val = null; attr. put(“A”, o); found = attr. contains. Key(“A”) ; if (found) { val = attr. get(“A”); attr. remove(“A”); } return val; o

Linearizability remove. Attribute(“A”) { Attribute val = null; attr. put(“A”, o); null found = attr. contains. Key(“A”) ; if (found) { val = attr. get(“A”); attr. remove(“A”); o attr. remove(“A”); } return val; o attr. put(“A”, o); attr. remove(“A”); null o remove. Attribute(“A”) { Attribute val = null; found = attr. contains. Key(“A”) ; if (found) { return val; null attr. put(“A”, o); null attr. remove(“A”); o attr. put(“A”, o); null remove. Attribute(“A”) { Attribute val = null; found = attr. contains. Key(“A”) ; if (found) { val = attr. get(“A”); attr. remove(“A”); } o return val; attr. remove(“A”); null

Challenge Testing the linearizabiliy of composed operations … … … ……

remove. Attribute(“A”) { Attribute val = null; attr. put(“A”, o); found = attr. contains. Key(“A”) ; if (found) { val = attr. get(“A”); attr. remove(“A”); } return val; • Reconstruction in TOMCAT extremely challenging – Large traces – Large number of traces – Bugs occur in rare cases with specific key values

Our Solution Modularity Generates simple traces Enables Env control Base linearizability Restrict generated traces Influence Restrict generated traces

Modular Checking remove. Attribute(“A”) { Attribute val = null; attr. put(“A”, o); found = attr. contains. Key(“A”) ; if (found) { val = attr. get(“A”); attr. remove(“A”); } return val;

Challenge Testing Linearizability of composed operation in a modular fashion … … … ……

remove. Attribute(“O”) remove. Attribute(“GGG”) remove. Attribute(“LL”) remove. Attribute(“G”) remove. Attribute(“A”) remove. Attribute(“B”) remove. Attribute(“K”) remove. Attribute(“R”) remove. Attribute(“P”) remove. Attribute(“L”) {{ { Attribute val = null; found==attr. contains. Key(“B”) attr. contains. Key(“A”) attr. contains. Key(“K”); attr. contains. Key(“O”) found === attr. contains. Key(“GGG”) attr. contains. Key(“LL”) attr. contains. Key(“G”) attr. contains. Key(“K”) attr. contains. Key(“R”) attr. contains. Key(“P”) attr. contains. Key(“L”) if (found) { if return (found) val; { attr. get(“O”); val = val attr. get(“R”); return val = val attr. get(“GGG”); ===val; attr. get(“LL”); attr. get(“G”); attr. get(“K”); attr. get(“P”); attr. get(“L”); attr. remove(“O”); attr. remove(“GGG”); attr. remove(“LL”); attr. remove(“G”); attr. remove(“K”); attr. remove(“P”); attr. remove(“L”); attr. remove(“R”); } return } val; return val; attr. put(“O”, attr. put(“A”, o’); attr. put(“GGG”, attr. put(“K”, attr. put(“P”, attr. put(“LL”, attr. put(“G”, attr. put(“R”, o); o’); o); attr. put(“K”, o’); attr. remove(“R”);

Influence Base Environment remove. Attribute(“A”) { Attribute val = null; attr. put(“A”, o); found = attr. contains. Key(“A”) ; if (found) { val = attr. get(“A”); attr. remove(“A”); } return val;

Running Example Attribute remove. Attribute(String name){ Attribute val = null; found = attr. contains. Key(name) ; if (found) { val = attr. get(name); attr. remove(name); } return val; }

remove. Attribute(“A”) { Attribute val = null; attr. put(“A”, o); found = attr. contains. Key(“A”) ; if (found) { val = attr. get(“A”); attr. remove(“A”); } return val; Attribute remove. Attribute(String name){ Attribute val = null; found = attr. contains. Key(name) ; if (found) { val = attr. get(name); attr. remove(name); } return val; }

remove. Attribute(“A”) { Attribute val = null; null attr. put(“A”, o); found = attr. contains. Key(“A”) ; if (found) { val = attr. get(“A”); o attr. remove(“A”); } return val; o attr. put(“A”, o); attr. remove(“A”); null o remove. Attribute(“A”) { Attribute val = null; found = attr. contains. Key(“A”) ; if (found) { return val; null remove. Attribute(“A”) { Attribute val = null; found = attr. contains. Key(“A”) ; if (found) { null return val; attr. put(“A”, o); attr. remove(“A”); null o attr. put(“A”, o); null remove. Attribute(“A”) { Attribute val = null; found = attr. contains. Key(“A”) ; if (found) { val = attr. get(“A”); attr. remove(“A”); } o return val; attr. remove(“A”); null

COLT program CO extractor library spec Timeout candidate COs CO key/value driver influence driver instrument linearizability checking Non-Lin Execution

Benchmark • Used simple static analysis to extract composed operations – 19% needed manual modification • Extracted 112 composed operations from 55 applications – Apache Tomcat, Cassandra, My. Faces – Trinidad, etc… • Extracted all composed operations per application • We did not find additional composed operations – Using Google Code and Koders

112 Unknown

59 Non Linearizable 53 Unknown

17 Open Non Linearizable 42 Non Linearizable 53 Unknown

17 Open Non Linearizable 42 Non Linearizable 27 Linearizable 26 Globals

27 Linearizable 85 Non-linearizable

Easy Detection Attribute remove. Attribute(String name){ Attribute val = null; found = attr. contains. Key(name) ; if (found) { val = attr. get(name); attr. remove(name); } return val; }

Current Work • Define a class of data-independent composed operations • Use small model reduction to prove linearizability of dataindependent composed operations • Empirical study for the ratio of real life data-independent composed operations

Summary • • Writing concurrent data structures is hard Employing atomic library operations is error prone Modular linearizability checking Leverage influence • Sweet spot – Identify many important bugs together with a traces showing and explaining the violations – Hard to find – Simple and efficient technique