Deep Learning Type Inference Vincent Hellendoorn 1 2

Deep Learning Type Inference Vincent Hellendoorn 1, 2, Chris Bird 2, Earl T. Barr 3 and Miltos Allamanis 4 1 UC Davis, 2 MSR Redmond, 3 UCL, 4 MSR Cambridge Grant #1414172 Exploiting the Naturalness of Software

Language Wars

Popular Vote

Challenger Java. Script Type. Script

Challenger Java. Script Type. Script

Civic Duty Java. Script

Civic Duty Java. Script Formal

Civic Duty Java. Script Heuristics

Civic Duty Java. Script Heuristics

Civic Duty Java. Script Learn it

Think Backwards Java. Script Type. Script

Specs Non-goals • Deep learning breakthrough • “Solve” type inference Goals • Powerful learning approach • Complement the compiler

Battleplan

Data

Exempli Gratia Type. Script

Exempli Gratia Type. Script

Exempli Gratia Type. Script Java. Script/Type. Script

Mining Expedition Types only Top 1000 Type. Script Repositories Compile & Filter Aligned Type-stripped code

Mining Returns Category Train Held-out Test Projects 620 78 78 Tokens 18 M 3. 9 M 1. 9 M

Mining Returns 1. any 2. string 3. number 4. void 5. boolean

Model

Tag! https: //i. stack. imgur. com/6 pd. IT. png

Tag! - num - - let a = 1 ; let b = a + 1 ;

Tag! - num - - - Project to Types Bi-Directional GRU Embedding let a = 1 ; let b = a + 1 ;

Tag! - num - - - Project to Types Bi-Directional GRU Embedding let a = 1 ; let b = a + 1 ;

Results

Battlegrounds Real annotations • Vs. naïve baseline, compiler Pure Java. Script • Vs. JSNice

Results

Results – Annotations Deep. Typer Basic RNN Baseline Compiler 0% 20% 40% 60%

Results – Annotations Deep. Typer Plain RNN Baseline Compiler 0% 20% 40% 60%

Results – Annotations Deep. Typer Plain RNN Baseline Compiler 0% 20% 40% 60%

Results – Annotations Deep. Typer Plain RNN Baseline Compiler 0% 20% 40% 60%

Results 100% Precision 80% 60% 40% Any confidence 20% 0% 0% 20% 40% 60% Recall 80% 100%

Results 100% Precision 80% 60% 40% 20% 0% 0% 20% 40% 60% Recall 80% 100%

Results 100% Precision 80% 60% 40% 20% 0% 0% 20% 40% 60% Recall 80% 100%

Results 100% Precision 80% 60% 40% 20% 0% 0% 20% 40% 60% Recall 80% 100%

Results – JSNice • Java. Script Only • 20 functions, 167 annotations

Results – JSNice Deep. Typer 50% 60% Correct 70% 80% 90% 100%

Results – JSNice Deep. Typer 0% 20% 40% Correct 60% Incorrect 80% 100%

Why not Both?

Results – JSNice Deep. Typer Hybrid 0% 20% 40% Correct 60% Incorrect 80% 100%

Outcomes What did we do? • Train for type inference • Highly complementary What’s next? • Towards full type inference • Interact with Compiler

Questions? What did we do? • Train for type inference • Highly complementary What’s next? • Towards full type inference • Interact with Compiler