For: intermediate front-end developers and expert developers that need a quick reference.

There are lots of ways to get your front-end architecture to the client. Usually the basic concepts seem easy and quickly done, but you will soon find yourself

make an effort to integrate it in a running environment,
make it work for the world (browser compatibilities and such)
handle external dependencies
write code that’s understandable for your colleagues (and yourself in 6 months)

Yep you’ve guessed it, with every demand you put on your code, the effort will go up exponentially.

This talk is not about how to manage your project. There’s many good books (e.g. Eric Ries – The Lean Startup) and articles to be found about this. This talk focusses on some core steps that I generally like to make to get an optimal work environment that gives me the least amount of friction during development.

During development you will work with lots of individual files, comments, testers and such, to ensure that the project is structurally sound, but also understandable for humans. But as soon as we deploy for our client, there will only be machines interpreting your code, and we’d like to get rid of all that isn’t strictly necessary and really get the highest performance we can get.

Assumptions

I have to make some assumptions about your environment, otherwise I would have to go all the way back to installing linux. So the things that I assume are:

you are devloping / deploying on linux machines under the debian architecture (I use Ubuntu)
you have NVM (Node Version Manager) installed (https://github.com/creationix/nvm). I don’t include NVM in the build process, since it is installed with a shell script and I consider piping a curl-response to bash as a potential hazard for your organization.
you have basic knowledge of linux, client-server over http and javascript
you have installed sass

Versions

One issue that is persistent over the years, is versions of dependencies. Sometimes a dependency gets updated and sub-dependencies cannot be resolved anymore. A solution for that resides in the combination of NVM, NPM and Bower.

You might notice that I refrain from using global modules. I do this to detach the project as much as possible from anything that is, or should be available on your machine. This way, we can also ensure that we use the correct version, and not an unknown version that is set globally.

Node

We first need to define which version of Node we’d like to use. Usually, at the time of developing we’d like to use the latest stable.

╭─tim@The-Incredible-Machine ~
╰─➤ nvm install stable
Downloading https://nodejs.org/dist/v7.0.0/node-v7.0.0-linux-x64.tar.xz...
######################################################################## 100,0%
WARNING: checksums are currently disabled for node.js v4.0 and later
Now using node v7.0.0 (npm v3.10.8)

Here you see my machine pulling in the latest version of Node, which at this time is 7.0.0.

You can see which versions are available on your machine like this:

╭─tim@The-Incredible-Machine ~
╰─➤ nvm ls
   v5.5.0
   v6.8.0
-> v7.0.0
system
node -> stable (-> v7.0.0) (default)
stable -> 7.0 (-> v7.0.0) (default)
iojs -> N/A (default)

And you can switch between them like this:

╭─tim@The-Incredible-Machine ~
╰─➤ nvm use 7.0.0
Now using node v7.0.0 (npm v3.10.8)

Now we have Node in place, it can supply us with tools that we use to build our solutions with. We’ll first need to create a project.

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master›
╰─➤ npm init
This utility will walk you through creating a package.json file.
It only covers the most common items, and tries to guess sensible defaults.

See `npm help json` for definitive documentation on these fields
and exactly what they do.

Use `npm install <pkg> --save` afterwards to install a package and
save it as a dependency in the package.json file.

Press ^C at any time to quit.
name: (build-process)
version: (1.0.0)
description: Example build process
entry point: (index.js)
test command:
git repository: (https://github.com/timmeeuwissen/build-process.git)
keywords:
author: Tim Meeuwissen
license: (ISC)
About to write to /home/tim/Git/build-process/package.json:

{
 "name": "build-process",
 "version": "1.0.0",
 "description": "Example build process",
 "main": "index.js",
 "scripts": {
 "test": "echo \"Error: no test specified\" && exit 1"
 },
 "repository": {
 "type": "git",
 "url": "git+https://github.com/timmeeuwissen/build-process.git"
 },
 "author": "Tim Meeuwissen",
 "license": "ISC",
 "bugs": {
 "url": "https://github.com/timmeeuwissen/build-process/issues"
 },
 "homepage": "https://github.com/timmeeuwissen/build-process#readme"
}


Is this ok? (yes)

Node is basically a javascript-engine running in a linux environment (e.g. on a server). There are lots of great tools written in node to interpret and mutate your project’s files to become client-friendly.

Bower

Bower is one of these tools. It enables to get dependencies for your front-end architecture. Whenever you feel like using jQuery, lodash, react, material-design or whatever you prefer to use, you will always need to get the dependencies, structure them in a certain way and keep track of their versions. Bower is NPM for front-end related components and does exactly that.

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ npm i bower --save-dev
build-process@1.0.0 /home/tim/Git/build-process
└── bower@1.7.9

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ node_modules/.bin/bower init
? name build-process
? description Example build process
? main file index.js
? keywords
? authors Tim Meeuwissen
? license ISC
? homepage https://github.com/timmeeuwissen/build-process
? set currently installed components as dependencies? Yes
? add commonly ignored files to ignore list? Yes
? would you like to mark this package as private which prevents it from being accidentally published to the registry? Yes

{
 name: 'build-process',
 description: 'Example build process',
 main: 'index.js',
 authors: [
 'Tim Meeuwissen'
 ],
 license: 'ISC',
 homepage: 'https://github.com/timmeeuwissen/build-process',
 private: true,
 ignore: [
 '**/.*',
 'node_modules',
 'bower_components',
 'test',
 'tests'
 ]
}

? Looks good? Yes

TypeScript

I rather use typescript than I use plain JS. Typescript is a superset of JS, that needs to be pulled through a compiler in order for it to be able to run on a client. There are always reasons not to do stuff, but I’d like to share my reasons to do it anyhow.

It is developed and maintained by Microsoft. This isn’t the smallest guy in town, and they do an excellent job at it.
It is a superset, and depending on your settings you can or cannot use typing wherever you want (May I kindly request you enforce every variable to be typed strictly, for reasons to follow)
Code-completing gets a hell of a lot more fun for your IDE (I personally really like Visual Studio Code on Linux from Microsoft. It’s free, try it!)
You can always work in the latest standards. Depending on your compiler arguments the code will be transpiled to any given EcmaScript standard you require for your company. The TypeScript compiler nicely polyfills whatever isn’t available in that ES version, and as time moves on, browsers get better, and your code will deprecate less fast (you can export to ES6 on any given day of the week e.g.).
You can more easily apply your backend architecture skills when you work with the latest ES version.
Your fellow developers will exactly know the input and output of each and every function without knowing the intricate details of your application.

Packing and transpiling

It’s a safe bet that we will create lots of documents that will all depend on each other. Browserify is able to follow these dependencies and combine them in to one file. There is a plugin called tsify, which basically runs the typescript compiler on the files before or after the merging has happened.

Lets add it to our project:

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ npm i browserify tsify typescript --save-dev
build-process@1.0.0 /home/tim/Git/build-process
├─┬ browserify@13.1.1
│ ├── assert@1.3.0
…
…

Get the google closure compiler. We will use this to pipe the output of browserify through.

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ npm i google-closure-compiler --save-dev
build-process@1.0.0 /home/tim/Git/build-process
└─┬ google-closure-compiler@20161024.1.0

Typings

Now, not every project is written in TypeScript, but we would like to be able to rely on their behavior within our files. E.g. jQuery might return some structure after invoking it, and we want to be able to recognize that output and work with it as such. Typings is a library filled by lots of wonderful people with interfaces of these external dependencies, so you don’t have to do it yourself!
Lets get it first:

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ npm i typings --save-dev 1 ↵
build-process@1.0.0 /home/tim/Git/build-process
└─┬ typings@1.5.0
├── archy@1.0.0
…
…
╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ node_modules/.bin/typings init

Sass

It’s important that every part of our application is structured in such a way that someone else understands what he or she is looking at. This includes the style. Style documents get easily overlooked and deemed as less important, but in my experience they are often responsible for the biggest part of the technical debt. Files with thousands of lines of style that react with the html, and no way to understand or properly refactor aren’t an uncommon sight.

In a separate document I plan to get deeper in how you can structure in such a way that you won’t build your own little jungle of css. Here, I will remain to focus on the build-steps and high-level reasoning.

Sharing of configuration and building

It often happens that some variables are shared across front-end architecture. Examples might vary between a path to the CDN or a max-amount of items within a caroussel.

Assuming that you already have sass installed, I could encourage you to also install sass-json-vars.

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ sudo gem install sass-json-vars

Because we always want to compile our files this way, it’s handy to create a helper that helps with picking up all scss files that match a pattern, and converts them to a normal css file.

I created a directory helpers with the file build_css.sh that basically contains this:

#!/bin/bash

# $1 = directory to scan for documents
# $2 = directory to put finished css documents in

find $1 -name [^_]*.scss 2> /dev/null | while read input
do
  output=$(echo $input | sed s@^$1@$2@ | sed s@\.scss$\@\.css@)
  outputdir=${output%/*}
  mkdir -p $outputdir
  sass -r sass-json-vars -t compressed $input ${output}
done

Call it with an input and an output dir as first and second argument. Nice helper right?

p.s. don’t forget to make it executable.

Basic file structure

Now that we have our external dependencies in, our directory structure should look like this:

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ tree
.
├── bower.json
├── helpers
│   └── build_css.sh
├── package.json
└── typings.json

(I’ve omitted some documents for the tree-view, you can do this yourself also by creating the alias in your .bashrc: alias tree=”tree -C -I ‘vendor|node_modules|bower_components'”)

Lets add some extra folders to give some directions where stuff should go.

First all public stuff.

Here’s where all scripts that can be visited by a browser will reside. All compiled files will be written to these directories. Why not dist? Since these projects are intended to be a dependency by other scripts, other parts like sass or typescript files are also part of the stuff that’s distributed, but those should never go ‘public’ so there you have it :-).

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ mkdir -p public/api public/css public/js

Now the source stuff.

There will occur situations in which you want to have a special interface for an external dependency. Typings will create its own dir, but here we have a space in which we can put our own custom ones.

Sass is split in multiple files which don’t have to be converted individually (partials) and functions that cover some visual logic (mixins).

The conf dir will have ‘constants’ variables that will become fixated in the code at compiletime.

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ mkdir -p sass/partials sass/mixins ts helpers typings_local conf

By now our structure should look like this:

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ tree
.
├── bower.json
├── conf
├── helpers
│   └── build_css.sh
├── package.json
├── public
│   ├── api
│   ├── css
│   └── js
├── sass
│   ├── mixins
│   └── partials
├── ts
├── typings.json
└── typings_local

11 directories, 4 files

Create a .gitignore. It makes it so much nicer if you don’t have to store all external dependencies in your own repo. Notice that all files in public are checked in as normally. Optionally you could run a post-commit hook that builds the project so you always know that the built files represent the state of the original source files.

node_modules
bower_components
typings
.sass-cache
ts/**/*js

Tying it together

There are so so so many ways to run tasks. You can use Grunt, Gulp or all kinds of fancy task-runners. Using task-runners can bring many advantages. But for me, a build process is something that should be intuitive whether or not you are familiar with a project or even a programming language. Linux has a common make process, and in my opinion whatever you want to expose as build steps should go through that mechanism. Makefile.

So if you decide on going with plain bash scripts, Grunt, Gulp or whatever you like, always make sure that your endpoints are also mapped in your makefile. In this way you can reliably build all your projects – but also detect issues on all your projects – in the same way, no matter what’s running underneath the hood of the project.

Since there isn’t any real exciting stuff going on in this project, we already have to do a makefile, I see no reason to already start implementing one of these task-runners.

Let’s make a Makefile:

.PHONY: all clean get-deps build build-js build-css

all: get-deps build

clean:
    -rm public/css/*.css
    -rm public/js/*.js
    -find ts/ -name "*.js" -type f -delete

get-deps:
    nvm install 7.0.0
    nvm use 7.0.0
    npm i
    node_modules/.bin/bower i
    node_modules/.bin/typings i
    sudo gem install sass-json-vars

build: build-js build-css
build-js:
    node_modules/.bin/browserify -p [ tsify --target es3 ] ts/app.ts \
        | java -jar node_modules/google-closure-compiler/compiler.jar \
        --create_source_map public/js/app.map --source_map_format=V3 \
        --js_output_file public/js/app.js
build-css:
    helpers/build_css.sh sass public/css

serve:
    node_modules/.bin/static-server -i index.htm public

I’ll explain briefly what happens

Make clean clears the public folders, since they can be regenerated. It also removes .js files in the ts folder. Some IDEs create these files to test their validity.

Make get-deps gets the dependencies for the project. This can be ran at your test and merge servers every time before building.

Make build builds the JS from TypeScript, drags it through browserify to create one file and drags it again through the google closure compiler to garble it and optimize it. Once done, it creates the CSS from SASS

Make serve starts a super-simple server that enables you to test this front-end application visually on-screen.

This small server that helps you during development can be very handy. For now we don’t require any fancy rewriting, proxying or server-sided processing, so staticly serving assets should suffice. Install the package by running:

╭─tim@The-Incredible-Machine ~/Git/build-process ‹master*›
╰─➤ npm i static-server --save-dev
build-process@1.0.0 /home/tim/Git/build-process
└─┬ static-server@2.0.3
├─┬ chalk@0.5.1
…
…

Seeing it work

Lets populate the project with some base values in order to test the build process.

Configuration of the app that’s shared between css and js.

conf/app.json

{
  "color": "blue"
}

Entrypoint for css

sass/app.scss

@import '../conf/app.json';
@import 'partials/_example';

Set the background color to the color in the variable, and center the color name as text on the page

sass/partials/_example.scss

html,
body {
 width: 100%;
 height: 100%;
 line-height: 100%;
 background-color: $color;
 text-align: center;
 font-size: 40vw;
}

An interface to define what can be expected from the configuration.

ts/i_config.ts

interface IConfig {
 color: string
}

export default IConfig;

A basic javascript file that replaces the content of the body element to show that the config variable “color”

ts/app.ts

/// <reference path="../typings_local/require.d.ts" />

import IConfig from "./i_config"

let config = <IConfig>require("../conf/app.json");


window.onload = () => {
 document.body.innerHTML = config.color;
}

In order to load the external JSON file:

typings_local/require.d.ts

declare var require: {
 <T>(path: string): T;
 (paths: string[], callback: (...modules: any[]) => void): void;
 ensure: (paths: string[], callback: (require: <T>(path: string) => T) => void) => void;
};

The html file that combines it all together

public/index.htm

<!DOCTYPE html>
<html>
<head>
 <meta charset="UTF-8">
 <title>Build Process</title>
 <link rel="stylesheet" href="css/app.css">
 http://js/app.js
</head>

<body>
 JS not loaded
</body>

</html>

now run:

make clean build serve and see what happens!

What happens afterwards

This sets a base, but it’s far from done. And since it’s my first blogpost, I’ll first need to assess how this will go.

I plan on writing on lots of topics, but in sequel and relation to this post I’m considering stories about:

ServiceWorkers, PWA.
- TypeScript
- Caching
- Cache manipulation
TDD, BDD automated testing
- karma
- browserstack
- jasmine
- chimp

Let me know what you think!

p.s. you can find this code at https://github.com/timmeeuwissen/build-process

Ministry Of Dev

Techtalk about whatever comes to mind, by Tim Meeuwissen

Creating a resilient Front-End build process

Assumptions

Versions

Node

Bower

TypeScript

Packing and transpiling

Typings

Sass

Sharing of configuration and building

Basic file structure

Tying it together

Seeing it work

What happens afterwards

Like this:

Related

Assumptions

Versions

Node

Bower

TypeScript

Packing and transpiling

Typings

Sass

Sharing of configuration and building

Basic file structure

Tying it together

Seeing it work

What happens afterwards

Share this:

Like this:

Related