Behavioral tests / BDD with TypeScript

For: developers, architects and teamleads that want to incorporate unit-testing for their TypeScript projects

A couple of blog posts ago we’ve set up a basic build-line, in particular for TypeScript. In this post we’ll get our hands-on again and apply some automagic stuff for doing BDD and / or behavioral-testing on our builds.

note: this post only deals with the ‘how‘, not the ‘why‘ and ‘when‘. Read this if this has your interest.

Setting up the environment for Behavioral Testing

Lets start with setting up a testsuite.

We usually need stuff like

  • something that connects to a browser
  • something that runs the tests
  • something that interprets the tests
  • something that compiles all needed files

There is this wonderful package called chimpjs that already helps us out on most of these facets.  It does so by integrating and sprinkling magic over the following tools:

Let’s install it and see from there.

╭─tim@The-Incredible-Machine ~/Git/build-process ‹BDD› 
╰─➤ npm i chimp ts-node --save-dev
╭─tim@The-Incredible-Machine ~/Git/build-process ‹BDD*› 
╰─➤ ./node_modules/.bin/typings i cucumber chai --save-dev --source=dt

Configuring Chimp

Let’s set up chimp. Chimp is primarily a wrapper and seamless integration of multiple test frameworks, so it might not come as a surprise that we can set config options to these individual frameworks. By default the configuration options will be as such:

https://github.com/xolvio/chimp/blob/master/src/bin/default.js

These options can be overridden in our own file, and we have to, because chimp by default isn’t set up to use TypeScript.

Create a file chimp.conf.js.

module.exports = {

  // - - - - CUCUMBER - - - -
  path: './feature',
  compiler: 'ts:ts-node/register'

};

Extending the Makefile

Add your test-routine to the makefile

.PHONY: [what was already in there] test bdd

and add the rules (extend test if you’ve also done the tdd post):

 

test:
    node_modules/.bin/chimp chimp.conf.js --chrome

bdd:
    node_modules/.bin/chimp chimp.conf.js --watch

Let’s also create the proper directories

╭─tim@The-Incredible-Machine ~/Git/build-process ‹BDD*› 
╰─➤ mkdir -p feature/step_definitions

Create some tests

In order for us to know if we’ve properly set up the test framework, we want to create some tests. Since we’ve already created some nonsense during the creation of the generic build process, we’ll continue on that.

First create the .feature file

The feature file should tell in plain English what feature we expect, and how it behaves in different scenarios.

in: features/config.feature

@watch @feature

Feature: Seeing the effect of the config on the screen
  In order to know if the config was correctly applied,
  As a Developer
  I want to test some of the aspects of the config on the screen

  Scenario: Check if background color is corect
    Given the config has the color set to blue
    When we look at the website
    Then I should be having a blue background

Then we write implementation for this feature

The featuretest as written, can not directly be interpreted by our test framework. Our script just doesn’t know what ‘background color’ means, and what element is been intended to check. So that’s why we create support for these steps. The nice thing is, that you might notice some punch holes in the sentences. Like ‘blue’, might be switched for another color, and ‘background’ might be ‘font-color’ or something along these lines. If you cleverly analyse your scenarios, you might become able to recognise standard patterns that you can re-use.

Be careful! A common caveat is that you start writing a language processor. Don’t do it! Tests should:

  • be straightforward
  • be easy to understand
  • have no deep connections with other tests 

Here’s the example implementation of the feature scenario. Put it in feature/step_definitions/config.ts

/// <reference path="../../typings_local/require.d.ts" />
import IConfig from "../../ts/i_config"

let config = <IConfig>require("../../conf/app.json");

export default function() {

  this.Given(/^the config has the color set to ([^\s]+)$/, function (color: string) {
    if (config.color !== color) {
      throw "Color in config mismatches the test scenario";
    }
  });

  this.When(/^we look at the website$/, function () {
    this.browser.url('http://localhost:9080');
    return this.browser.waitForExist('body', 5000);
  });

  this.Then(/^I should be having a ([^\s]+) background$/, function (color: string) {
    let browserResponse = this.browser.executeAsync(function(color: string , done: (response: boolean) => void) {
 
      let compareElem = document.createElement("div");
      compareElem.style.backgroundColor = color;
      document.body.appendChild(compareElem);
 
      let bodyElem = document.querySelector('body');

      done(
        window.getComputedStyle(compareElem).backgroundColor == window.getComputedStyle(bodyElem).backgroundColor
      );
    }, color);

    if (!browserResponse.value) {
      throw "BackgroundColor didn't match!";
    }

  });

}

Running the tests

By now we have set up TDD and BDD tests with TypeScript.  A simple

╭─tim@The-Incredible-Machine ~/Git/build-process ‹BDD*› 
╰─➤ make test

Should give you something like this:

Conclusion and notes

We are now fully able to write our tests – feature as well as function – in TypeScript, and have integrated them in our example build-process. We can run these tests on our own machine to verify our project locally.  BDD and TDD are set up separately so that we have more grip on either of the testing solutions and prevent coupling where not needed.

We are however not completely done yet.

  • We will have to set up some CI / CD make-tasks that can be ran at a headless server since we now leverage the browser in our own OS.
  • We will need make sure our watchers and compilers are set up properly, in order for BDD and TDD to run nicely in the back while developing our code.

We will go more in-depth on those aspects when we start hooking our project up to nginx and really start developing an application.

Changes applied in this blog post can be found at github.

Suggestions, comments or requests for topics? Please let me know what you think and leave a comment or contact me directly.

Why and When to do Behavioral or Test Driven Development (B/T)DD

For: Teamleads, Architects, Entrepreneurs and QA members that are searching for a path to higher quality.

Generally, testing is perceived as boring, time consuming and ‘expensive’. This is also the first question business people will ask when you propose it. ‘Fine, but how much does it cost?’. This article should give you some sort of a hand-hold to determine what it might bring you and if the time is right.

How to test your stuff, isn’t as valuable if you don’t have a solid understanding of the Why and When. I have written a post on how to implement TDD in a TypeScript build-process, about the how. Now it’s time for the reasons behind it.

When to start with automated testing

It’s in my opinion foolish to immediately start with writing tests when you start with creating a new product. Often you really don’t know what the product will become (you might think you do, but really, usually you don’t, read about this in The Lean Startup from Eric Ries). There will be many iterations of pivoting that will cause your application to go a completely different way when you build an MVP (Minimally Viable Product).

But once the product found it’s way in to the market, and the goals become more and more long-term, your focus will start to shift from creating new stuff, to make sure we create the good stuff. Code gets refactored all the time to be more performant and better readable. But stuff will break all the time.

At this moment your code will – and should – be tested to maintain a certain level of quality. This is the moment Automated Testing steps in.

Why would I do automated testing

I mean, developers know what they’ve done right? They can check what they’ve created?

That’s the general over-simplification we hear. It’s true in some sense, and happens always to reach your Acceptance Criteria, but it won’t suffice. With automated tests you can:

  • Run automated tests before merging to stable to know you are safe and automated rollout to staging or even production (Continuous Integration / Deployment)
  • Test against hundreds of browsers and their versions, on different devices and operating systems
  • Prevent regression
  • The tests define functionality. They are the place you can go to to find an example of integration
  • Do code coverage checks that give you information on how much of your code is covered by tests.
  • They cut time and reduce the risks that you have when going to production. It builds in a certain amount of certainty. Whenever a bug does slip in, you can write new tests to check for that issue. This makes sure that the exact bug doesn’t re-occur (hence regression tests)

What does the T/B DD stand for?

TDD means Test Driven Development

When you read it carefully, you see “Driven Development” trailing the first word “Test”.  This basically means: write your tests before you start writing any code at all. So, what are the benefits of doing this?

  • By writing your test first, you’ll have to think about the thing you are creating. Think about the outlines, the how and what. Since you are still not really hands-on, you also don’t have to improvise and be burdened by hacking stuff in to make it work. You just think about the feature or the unit that you want to add to the system. This makes your code mode atomically correct once you start writing and you’ll spot issues before they arise.
  • Now we know wat we can expect from the thing we want to create, we start running the test. The test will fail since the code for your new test isn’t there yet.
  • You iterate over your code to make your tests green, and if needed add other tests if the functionality isn’t sufficient yet, and start this process all over again.
  • You’ll deliver code that is restricted to what’s asked from it, not what future questions might be. You’ll deliver code that works, and what others can rely on.  All features are documented.

TDD is often synonym for Unit Testing. Unit testing means, each unit of your code should be testable as a separate black-box apart from the complete system. You’ll see that you’ll be mimicking the file structure of the original project. Don’t merge the two together although this might be tempting! Tests should run separate from your production code. Your code should never rely on your tests. Your tests should only rely on units of your code.

BDD stands for Behavioral Driven Development

So the ‘Driven Development’ is exactly the same as for TDD, but in this case we don’t test units, but the sum of their outcome. These units combined create an experience for the user, and your scrum stories rarely contain AC’s with the granularity of a Unit. To properly test the code, we’ll have to do integral tests that measure if all criteria for AC’s are met.

With BDD we:

  • click through the website
  • expect functionality to be there
  • finish operations, like creating a basket and ordering
  • test if the outcome is as desired.
  • If not correct, take screenshot, log errors and alarm.

This means:

  • have all AC written in a structural way
  • test AC in human readable text against our production and test environment
  • develop a feature base (since we already keep track of AC) that informs the next person about the what, why and how of all features in our application.

I can advise ChimpJS to do this for you, while writing your tests in Cucumber syntax. This is friendly for as well as Business as Tech. Here’s a great example of how that would look like!

 

What does testing cost?

So to finish with the first thing that will be asked. Investing in automated testing does cost money, but a lot less than humans doing the same thing.

The question really is, how much can you afford to screw up?

If the answer to that is: I don’t mind, then don’t do it. Because you wouldn’t hire a human to test either.

If the answer is: I do mind a bit but I don’t want to invest too much, then make sure that all software that is tightly related to KPIs is automatically tested. You will see that over time it will give you more than it costs you.

If testing is already an integral respected part of your deployment routine but not digitized yet, I would say: read this article again and draw your own conclusions.

 

Testing isn’t giving you 100% assurance. Nothing does. But you can always try to become better at what you do, and with that idea in mind be sure that you structurally test whenever changes are being made. I once spoke with a CTO that had a complete division of testers, that wrote tests apart from the development teams. To my recollection both teams where about equal in size. What we should learn from this is: when there is much at stake, you must do more to make sure things go right.

How much is for you at stake?

Let me know what you think in the comments! Want more of this? Use the Poll on the right of the screen, comment or contact me!

Unit-tests / TDD with TypeScript

For: developers, architects and teamleads that want to incorporate unit-testing for their TypeScript projects

A couple of blog posts ago we’ve set up a basic build-line, in particular for TypeScript. In this post we’ll get our hands-on again and apply some automagic stuff for doing TDD and / or unit-testing on our builds.

note: this post only deals with the ‘how‘, not the ‘why‘ and ‘when‘. Read this if this has your interest.

Setting up the environment for unit testing

So what do we need:

Some testing framework (we go with Jasmine)

There are lots of unit-test tools (mocha, chai, sinon, etc) out there that are really good. I at this moment prefer Jasmine. It’s well documented, stays relevant, serves an atomic purpose, configurable and has plugins separated through the NPM repo.

Some orchestrator / runner

We need an orchestrator to launch our tests in browsers. We use Karma for this.

Some browsers

There’s so many you can use, but also should use. Karma facilitates that you can hook up your own browser (open multiple on multple machines if you want) to test with. If that’s too manual you can go with solutions like: PhantomJS, chrome automated testing with Selenium / Webdriver, or doing it through Browserstack and have it tested on multiple versions of multple browsers on multiple operatingsystems and their versions. Lucky you, te runner we chose (Karma) supports interfacing with all of these as part of your testline.

Some reporters

What would we need to get a bit of grip and feeling with our test-process.

  • spec – show the entire spec of the units
  • coverage – we want to know if we’ve actually covered most of our logic (again, why you would like to do this will be described in another article)

 

You convinced me, two thumbs up, let’s do this.

So our lovely NPM can help us quite a bit with this. Do as followed:

╭─tim@The-Incredible-Machine ~/Git/build-process ‹unit-tests*›
╰─➤ npm i jasmine-core karma karma-browserify karma-browserstack-launcher karma-jasmine karma-phantomjs-launcher karma-coverage karma-typescript-preprocessor phantomjs-prebuilt watchify karma-spec-reporter --save-dev

Next chapter.. ;-).

Instructing the runner

Karma needs to know a bit about what it should do when we’ve asked for a test.

karma.conf.js

module.exports = function (config) {
 config.set({
   basePath: '',
   frameworks: ['browserify', 'jasmine'],
   files: [
     'spec/**/*_spec.ts'
   ],
   exclude: [],
   preprocessors: {
     'spec/**/*.ts': ['browserify','coverage']
   },
   browserify: {
     debug: true,
     plugin: [['tsify', {target: 'es3'}]]
   },
   reporters: ['spec', 'coverage'],
   port: 9876,
   colors: true,
   logLevel: config.LOG_INFO,
   autoWatch: true,
   browserDisconnectTimeout: 1000,
   browserDisconnectTolerance: 0,
   browserNoActivityTimeout: 3000,
   captureTimeout: 3000,
   browserStack: {
     username: "",
     accessKey: "",
     project: "build-process",
     name: "Build-process test runner",
     build: "test",
     pollingTimeout: 5000,
     timeout: 3000
   },
   coverageReporter: {
     type: 'text'
   },
   customLaunchers: {
     ie10: {
       base: "BrowserStack",
       os: "Windows",
       os_version: "7",
       browser: "ie",
       browser_version: "10"
     },
     chrome: {
       base: "BrowserStack",
       os: "Windows",
       os_version: "10",
       browser: "chrome",
       browser_version: "latest"
     },
   },
   browsers: ['PhantomJS'],
   singleRun: false
})}

don’t forget to create the directory that is scanned for your _spec.ts files

 

Extending the Makefile

Add your test-routine to the makefile

.PHONY: [what was already in there] test tdd

and add the rules:

test:
    node_modules/.bin/karma start --single-run

tdd:
    node_modules/.bin/karma start

 

Getting definitions of Jasmine

Since your code is written in TypeScript, your tests preferably are also written in TypeScript. You’ll need some definitions of the capabilities of Jasmine in order to use it properly. Luckily the people of typings are geniuses and supplied such a definition for us!

╭─tim@The-Incredible-Machine ~/Git/build-process ‹unit-tests*›
 ╰─➤ node_modules/.bin/typings i jasmine --source="dt" --global
 jasmine
 └── (No dependencies)

 

Test if we can test

Oh boy that is a nice title :-). Let’s write some nonsense first, so we can write tests for it later.

The nonsense

Now create some simple example module like ts/example_module.ts:

type someCallback = (someString: string) => string;

export default class example_module {

  constructor(private someVar: string, private callback: someCallback) {

  }

  public some_method(){
    console.log('some method ran!');
  }

  public get_string(): string {
    this.some_method();
    return this.callback(this.someVar);
  }

}

 

There’s a range of nonsense that can be applied in even more bizarre ways that I don’t intend on pursuing  if you don’t mind. This should suffice 🙂

Let’s test this nonsense

Create this testfile in spec/example_module_spec.ts

Generally it’s a good idea to separate the tests from the project since they otherwise clutter the area you’re working in. But do try to mimic the structure that’s used in your normal ts folder. This allows you to find your files efficiently. We append _spec to the filename, because when your project grows, it’s not uncommon to create a helper or two, which shouldn’t be picked up automatically.

/// <reference path="../typings/index.d.ts" />

import ExampleModule from "../ts/example_module"

describe('A randon example module', () => {

  var RANDOM_STRING: string = 'Some String',
      RANDOM_APPENDED_STRING: string = ' ran with callback',

      callback = (someString: string): string => {
        return someString + RANDOM_APPENDED_STRING;
      },
      exampleModule: ExampleModule;

   /**
    * Reset for each testcase the module, this enables that results
    * won't get mixed up.
    */
   beforeEach(() => {
     exampleModule = new ExampleModule(RANDOM_STRING, callback);
     spyOn(exampleModule, 'some_method');
   });

   /**
    * testing the outcome of a module
    *
    * Should be doable for almost all methods of a module
    */
   it('should respond with a callback processed result', () => {
     let response = exampleModule.get_string();

     expect(response).toBe(RANDOM_STRING + RANDOM_APPENDED_STRING);
   });

   /**
    * testing that specific functionality is called
    *
    * You could make use of this, when you expect a module to call
    * another module, and you want to make sure this happens.
    */
  it('should have called a specific method each time the string is retrieved', () => {
    // notice that, because of the beforeEach statement, the spy is reset
    expect(exampleModule.some_method).toHaveBeenCalledTimes(0);

    // execute logic twice
    exampleModule.get_string();
    exampleModule.get_string();

    // expect that the function is called twice.
    expect(exampleModule.some_method).toHaveBeenCalledTimes(2);
  });
});

The result:

╭─tim@The-Incredible-Machine ~/Git/build-process ‹unit-tests*›
╰─➤ make test
node_modules/.bin/karma start --single-run
08 12 2016 22:58:35.109:INFO [framework.browserify]: bundle built
08 12 2016 22:58:35.115:INFO [karma]: Karma v1.3.0 server started at http://localhost:9876/
08 12 2016 22:58:35.116:INFO [launcher]: Launching browser PhantomJS with unlimited concurrency
08 12 2016 22:58:35.130:INFO [launcher]: Starting browser PhantomJS
08 12 2016 22:58:35.380:INFO [PhantomJS 2.1.1 (Linux 0.0.0)]: Connected on socket /#RxSPDX6Lu-LvxyP2AAAA with id 76673218
PhantomJS 2.1.1 (Linux 0.0.0): Executed 2 of 2 SUCCESS (0.04 secs / 0.001 secs)
--------------|----------|----------|----------|----------|----------------|
File          |  % Stmts | % Branch |  % Funcs |  % Lines |Uncovered Lines |
--------------|----------|----------|----------|----------|----------------|
 spec/        |      100 |      100 |      100 |      100 |                |
 app_spec.ts  |      100 |      100 |      100 |      100 |                |
--------------|----------|----------|----------|----------|----------------|
All files     |      100 |      100 |      100 |      100 |                |
--------------|----------|----------|----------|----------|----------------|

╭─tim@The-Incredible-Machine ~/Git/build-process ‹unit-tests*›
╰─➤

Or as my lovely console likes to tell me in more color:

build-process-tdd.png

 

Check the PR for the actual code.

Want more? Any ideas for the next one? Let me know or use the poll on the right side of the screen!