Skip to main content

Dry Runs

Since v1.0.0

Catch syntax and binding errors​

Dry runs can be performed on features to verify that they are syntactically correct and that all references have bindings that will resolve at runtime. This allows you to quickly validate features and catch potential errors before committing them to an execution cycle. The -n|--dry-run CLI option is used to launch Gwen in this mode.

The primary purpose of dry runs is to perform static validation of all syntax, bindings and StepDefs throughout your feature and meta files. In a dry run:

  • Browsers sessions are never started
  • Web pages are never loaded (and web elements can therefore cannot be accessed)
  • Non static bindings are never evaluated
  • JavaScript is never evaluated
  • System processes are never evaluated
  • Databases are never accessed

Results will be logged to the console and in HTML reports that you can inspect to find discovered issues.

Example​

If you leave out or forget to define a selector for an input field, it would be detected by a dry run and reported in the HTML report as follows:

Example dry run with error

DryRun Annotations​

Since v3.55.0

@DryRun annotations can be used on any step to assign a value (or values) to a named reference at dry-runtime and are very useful when:

  • You want to exercise a step definition over a fixed set of inputs
  • Or a binding cannot be evaluated and causes failures that otherwise would not occur outside of a dry run
info

DryRun annotations are only honoured by dry runs and are completely ignored otherwise.

tip

Use DryRun annotations sparingly or when necessary only.

Example​

Consider a meta file with the following step definitions:

meta

  @StepDef
  Scenario: I perform step 1
    Then number should be "1"

  @StepDef
  Scenario: I perform step 2
    Then number should not be "1"

Consider the following scenario where you might want to call the 1st or 2nd step definition depending on the value contained in some number displayed on a web page.

  Scenario: Perfom a step
    Given number can be located by css ".n"
     When I perform step ${number}

This would work perfectly fine when ${number} evaluates to 1 or 2 in a web page loaded in a browser. However during a dry run, browsers sessions are never started, web pages are never loaded and looking up web elements and fetching their values is therfore not possible. The 2nd step above would fail since the number in the web element is nowhere to be found. In this instance, the standard Gwen behavior is to return a dry run placeholder value of $[dryValue:webElementText]. That 2nd step would resolve to the following:

  When I perform step $[dryValue:webElementText]

This would result in an undefined step error being reported since it cannot be matched to any step definition in our meta.

A non-ideal workaround​

Before @DryRun annotations were introduced, one way to work around this problem involved introducing predicates and using those as guards on calls to each step definition explicitly as follows:

  Given number can be located by css ".n"
    And number 1 is defined by js "${number} == 1"
    And number 2 is defined by js "${number} == 2"
   When I perform step 1 if number 1
    And I perform step 2 if number 2
   Then ..

This works because Gwen always resolves predicates to true values in dry runs. However, this is not ideal because it forces you to break your abstraction and introduces unwanted redundancies. This is the primary reason why @DryRun annotations were introduced; to avoid this kind of workaround.

Single Values​

Annotation synax: @DryRun(name='<name>',value='<value>')

The solution is to provide a dry run value for the number reference in the step using the @DryRun annotation as follows:

  Scenario: Perform a step
    Given number can be located by css ".n"
     When I perform step ${number}    @DryRun(name='number',value='1')
     Then ..

Now when you do a dry run on this, the ${number} reference will take on the value 1 and resolve to the following step:

  When I perform step 1

This would result in a successful dry run of the 1st step definition.

Multiple Values​

Since v3.55.4

Annotation synax: @DryRun(name='<name>',value=['<value-1>','<value-2>','<value-N>'])

  • [ and ] interchangeable with { and }

What if we wanted to exercise both step definitions 1 and 2 in the meta? The solution is to provide a list of dry run values for number on the calling step using the @DryRun annotation as follows:

  Scenario: Perform a step
    Given number can be located by css ".n"
     When I perform step ${number}    @DryRun(name='number',value=['1','2'])
     Then ..

Now when you perform a dry run on this, the ${number} reference will take on the value 1 followed by 2 and the 2nd step above will be expanded in-place into the following two steps:

  When I perform step 1
   And I perform step 2

This will result in a successful dry run of both the 1st and 2nd step definitions. So now the entire meta will be exercised.

Delimited Values​

Annotation synax: @DryRun(name='<name>',value='<value-1> <value-2> <value-N>') (space delimited example)

Similarly, you can also assign a dry run value to any binding that cannot be evaluated during in dry run mode such as a JavaScript binding, for example.

Consider the following where you want to source a space delmited list of numbers from a script and loop over each one and call the step definition with that number.

meta

  @StepDef
  Scenario: I perform step 1
    Then number should be "1"
    
  @StepDef
  Scenario: I perform step 2
    Then number should not be "1"

  Scenario: Perform all steps
    Given numbers is defined by js "$('.n').toArray().join(' ')"
     When I perform step ${number} for each number in numbers delimited by " "
     Then ..

The call to the step in the loop on line 11 would fail in a dry run scenario, resulting in an undefined step error because it would resolve to the following:

   When I perform step $[dryValue:javascript]

In this instance, Gwen will have assigned the $[dryValue:javascript] placeholder value to the numbers binding on line 10 since it is defined with a JavaScript expression which cannot be evaluated at dry-runtime. This placeholder value would then used to feed the loop, and since it has no spaces in it, the entire value itself becomes the sole undelimited value.

The solution is to assign a space delimited dry run value of 1 2 to the numbers binding as follows:

 Given numbers is defined by js "$('.n').toArray().join(' ')"   @DryRun(name='numbers',value='1 2')
  When I perform step ${number} for each number in numbers delimited by " "
  Then ..

Note: If you're using the DocString synax, you can do it like this:

  Given numbers is defined by js    @DryRun(name='numbers',value='1 2')
        """
        $('.n').toArray().join(' ')
        """
    When I perform step ${number} for each number in numbers delimited by " "
    Then ..

Now when you perform a dry run, the numbers reference will take on the value 1 2 and both our step definitions will be called in the the loop:

  When I perform step 1
   And I perform step 2

This will result in a successful dry run of both step definitions.

note

You can also source values from settings in annotations using the ${setting.name} syntax in the value attribute, should you wish to externalise your dry run values.