UI Test Automation with Spec Flow Presented by

UI Test Automation with Spec. Flow Presented by Eugeny Govako For internal use only

Overview • • • Gherkin and test automation NUnit based automation approach Spec. Flow automation evolution Q&A For internal use only

Gherkin and test automation • • Test Automation is implementing specifications step by step using some programming language Gherkin is the language for executable specifications Gherkin itself is a kind of a programming language with well defined syntax Originally invented to use with Cucumber framework For internal use only

Prerequisites • Specifications (written in Gherkin preferably) • Test API • Driver providing access to Application with ability to find required control • Wrappers for controls • Application under test Gherkin Application Test. API For internal use only

Non-Spec. Flow automation • NUnit tests implementation based on Gherkin lines For internal use only

Non-Spec. Flow automation summary • No direct link between specification (Gherkin) and test code • Test code uses Test API layer that provides access to controls, allowing application interaction Gherkin Test. API Application For internal use only

Spec. Flow automation Specification linked to test Test development is very fast Automation code base grows even more fast Test Support trouble For internal use only

Spec. Flow automation: basics • Using parameters to reduce code amount For internal use only

Spec. Flow automation: basics • Tables can be used row-by-row For internal use only

Spec. Flow automation summary • Tests code is directly connected to Gherkin via Spec. Flow runner • Test API layer still used for providing controls • Binding method arguments reduce code amount Gherkin Spec. Flow engine Bindings Parameters For internal use only Test API Application

Gherkin requirements for automation • Reuse steps • Use parameterization where possible Any Gherkin line can be automated. However, there’s always tradeoff between effort needed to create scenario and effort needed to automate. For internal use only

Spec. Flow automation: advanced features • Using step argument transformations For internal use only

Spec. Flow automation: advanced features • Tables can be used in transformations • Tables can be converted to parameter maps For internal use only

Spec. Flow automation: generic approach • Trouble? • Parameterization leads to huge maps of controls • Conflicting Regex for same actions in different contexts • Code base still grows fast • How? • Execution Context – specify where to look for control • Use direct transformations to automation wrappers • Register controls with friendly names in a context-specific resolver • Silver Bullet? • Makes it possible to have bindings provided from the box, along with control wrappers For internal use only

Spec. Flow automation: control-based Example: working with buttons • IButton wrapper • Provides access to Visibility, enabled state, text, click functionality • Implementation comes from automation framework • Visual Resolver: • Allows registering button by friendly name • Allows resolving button in an active context • Automatically transforms control requested by friendly to DSL interface used inside binding method • Step. Argument. Transformation returning IButton • Using Visual Resolver, convert friendly name (string) into IButton wrapper instance • Button Bindings extending Control. Binding. Base<IButton> • Contain bindings that use string -> IButton conversion For internal use only

Spec. Flow automation: control-based Visual Resolver example For internal use only

Spec. Flow automation: control-based Control binding class example For internal use only

Control-based approach: summary • Uses Spec. Flow native step interpreter • Uses Regexp for steps and parameters mapping • Uses Visual Resolver to convert names to DSL Reduced number of steps Development costs are low Diagnostics is not good Similar steps still exist for different control types For internal use only

Spec. Flow automation: behavior driven Control based example: Scenario Block 4 binding Controlmethods What to Control friendly name type Behavior driven example Control wrapper from Visual Resolver Behavior control should implement For internal use only check

Behavior-driven approach Behavior Interface Property shown / not shown ICan. Be. Visible Is. Visible enabled / disabled ICan. Be. Enabled Is. Enabled has “some” text IHas. Visible. Text • Works with all controls supporting behaviors designated by atomic interfaces • New behaviors can be introduced easily For internal use only

Behavior driven approach: conclusion Automation bindings number significantly reduced Focus on behavior, not control (closer to business) Still risks of conflicts hard to resolve via regexp For internal use only

Romashka Driver • Give up Regexp! • Use smart step parser and grammar rules • Find controls and analyze/apply behaviors No Regexp conflicts New behaviors added easily Support new controls by applying behavior interfaces to DSL Diagnostics is even worse Hard to understand contribute Crush a fly upon a wheel feeling For internal use only

Specflow automation in Autobahn FX UI • 100% P 1 / 80% all automation coverage (of possible to automate) in 5 projects using Spec. Flow • More than 23000 test cases automated • Reasonable test support costs For internal use only

Q&A For internal use only