Having a suite testing a lot of features is a good thing. But if your project supports a lot of different Rubies – MRI-Ruby, JRuby etc. – you might face platform dependent bugs. Read on, if you like to know how we worked around some platform dependent bugs in
aruba to keep the build status “green” on all support Rubies.
Cucumber is quite a cool tool to build a test suite using a more or less natural language. I use it in some of my projects together with aruba – an extension for some popular testing frameworks to write tests for commandline applications. It supports different versions of MRI-Ruby and JRuby.
Maybe you also know the situation: You are the developer of an application/a library supporting different platforms like JRuby and/or MRI-Ruby on Linux, Windows and/or Mac OS X. You might have gotten into a pretty pickle adding a new feature whose tests only fails if you run them with JRuby.
If the reason for this failure is a bug in the platform – like jruby/ruby#3126 – the only thing you can do some times, is to skip the test until the bug is fixed. This article describes how you can skip tests in your
cucumber test suite both for cucumber 1.x and cucumber 2.x starting with the more easy to use version 2 (for this use case).
First, you need to add
aruba to your
source 'https://rubygems.org' gem 'cucumber', '~> 2.0.2' gem 'aruba', '~> 0.9.0.pre'
And run this command afterwards to install the gems.
Next, create a file named “features/support/env.rb”. This file should contain the following code.
After this, please create a feature-test named “features/convert_text_file.feature” with the following content:
Feature: Convert Text file Scenario: Existing input file Given an empty file named "test.in" When I successfully run `mycommand.sh` Then the file "test.db" should exist Scenario: Un-existing input file Given a file named "test.in" does not exist When I run `mycommand.sh` Then the exit status should be 1
Additionally create the executable
bin/mycommand.sh with the following content:
#!/usr/bin/env bash [[ -f test.in ]] && touch test.db
And make it an executable.
chmod +x bin/mycommand.sh
You can count yourself lucky if you can use
cucumber 2. This versions has a method to skip a test - aka “scenario”. You can use this method to skip it under certain conditions.
If you need to skip the scenario “Un-existing input file” on Windows, Mac OS X and Unix attach the following tags to it:
@ignore-platform-unix – normally this would be a single tag skipping only one platform.
Feature: Convert Text file Scenario: Existing input file Given an empty file named "test.in" When I successfully run `mycommand.sh` Then the file "test.db" should exist @ignore-platform-windows @ignore-platform-mac @ignore-platform-unix Scenario: Un-existing input file Given a file named "test.in" does not exist When I run `mycommand.sh` Then the exit status should be 1
Next, you need to create
Before-hooks for your supported platforms within
features/step_definitions/hooks.rb – the filename is arbitrary chosen – with the following content.
Before '@ignore-platform-windows' do skip_this_scenario end Before '@ignore-platform-unix' do skip_this_scenario end Before '@ignore-platform-mac' do skip_this_scenario end
cucumber and you will notice, that the tagged scenario is skipped.
bundle exec cucumber # => Feature: Convert Text file # => # => Scenario: Existing input file # => Given an empty file named "test.in" # => When I successfully run `mycommand.sh` # => Then the file "test.db" should exist # => # => @ignore-platform-windows @ignore-platform-mac @ignore-platform-unix # => Scenario: Un-existing input file # => Given a file named "test.in" does not exist # => When I run `mycommand.sh` # => Then the exit status should be 1 # => # => 2 scenarios (1 skipped, 1 passed) # => 6 steps (3 skipped, 3 passed) # => 0m0.122s
Unfortunately this only works for
cucumber 2.x. For
cucumber 1.x you need to use a hack. Both versions of
cucumber support the use of the
cucumber.yml – the
cucumber-configuration file. You can safely use ERB within this file.
cucumber 1.x, you need to modify your
Gemfile. Please replace the version of
1.3.20 and run
bundle install after that.
source 'https://rubygems.org' gem 'cucumber', '~> 1.3.20' # <- Change version here gem 'aruba', '~> 0.9.0.pre'
Additionally there’s a library called “FFI”. You typically use this library for programmatically loading dynamic libraries. But this library also comes with some methods to detect the platform you’re running your code on – e.g. Windows, Linux, Mac OS X. Make sure, that you add this library to your
Gemfile to make it available for your test-setup.
gem 'ffi', '~> 1.9.10'
Now run this command.
Combining both things –
ffi – you can build a solution for
cucumber 1.x which gives you the same result like the
#skip_this_scenario-method known from
<% require 'ffi' ignore_opts =  ignore_opts << "--tags ~@ignore-platform-java" if RUBY_PLATFORM.include? 'java' ignore_opts << "--tags ~@ignore-platform-mri" if !RUBY_PLATFORM.include? 'java' ignore_opts << "--tags ~@ignore-platform-windows" if FFI::Platform.windows? ignore_opts << "--tags ~@ignore-platform-unix" if FFI::Platform.unix? ignore_opts << "--tags ~@ignore-platform-mac" if FFI::Platform.mac? ignore_opts = ignore_opts.join(' ') %> default: --tags ~@wip <%= ignore_opts %> wip: --wip --tags @wip:3 <%= ignore_opts %>
After that run
cucumber. You will notice, that only one
scenario is run by
cucumber. The other one is simply ignored.
bundle exec cucumber # => Using the default profile... # => Feature: Convert Text file # => # => Scenario: Existing input file # => Given an empty file named "test.in" # => When I successfully run `mycommand.sh` # => Then the file "test.db" should exist # => # => 1 scenario (1 passed) # => 3 steps (3 passed) # => 0m0.111s
If you need to skip scenarios under certain conditions, you can use the
#skip_this_scenario-method within a
cucumber 2.x. If you need to use
cucumber 1.x, you can “build” your own solution by combining ERB within the
cucumber.yml together with
Thanks for reading.
DiscussionIf you found a mistake in this article or would like to contribute some content to this article, please file an issue in this Git Repository
DisclaimerThe contents of this article are put together to the best of the authors' knowledge, but it cannot be guaranteed that it's always accurate in any environment. It is up to the reader to make sure, that all information found in this article, does not do any damage to the readers's working environment or whereever this information is applied to. In no event shall the authors or copyright holders be liable for any claim, damages or other liability, arising from, out of or in connection with this article. Please also note the information given on the Licenses' page.
- Break it Baby, break it – How the network sniffer "Wireshark" can break Rubygems
- Having No Fun with Rubygems, Systemd, Docker and Networking
- Creating an IRB-based Read-Eval-Print-Loop-console for your project
- Setting up a Ruby-based HTTP-server to back your test suite
- Setting up an HTTP proxy with ruby