Terraform provider kong – fully tested using docker containers

A couple of posts ago I talked about how you could achieve full stack testing in Go using Docker containers. I have just finished the first version of terraform provider kong, a terraform provider for Kong that is built on top of gokong.

At the moment at Form3 we configure Kong using a custom Ruby script that runs on our Kong container and configures Kong upon boot up. Whilst this works there are a number of problems with this approach:

  • It is hard to know what the end state of Kong is as you end up with a whole bunch of Ruby code that is hard to follow
  • If you want to migrate the container over time it can be quite tricky as its hard to clean up APIs, consumers and other Kong resources as you delete them (the database Kong uses is long lasting)

By writing a custom terraform provider for Kong all of these problems go away.  As terraform uses a declarative language you simply declare the state that you want the thing you are configuring to be in (in this case Kong) and terraform looks at the current state and automatically works out how to get there.  This makes everything so much simpler.  A good example is if you remove an API in our current world you have to remember to write a line of Ruby to delete the old API.  When using terraform you simply remove the API from your configuration and terraform deletes it for you.

Building upon the way that I wrote gokong I wrote terraform provider kong in the same vein, ie full stack testing using real components and no mocks! This means that the acceptance tests in terraform provider kong spin up a real Kong container and apply terraform plans to it, check it is in the expected state and then destroy it again. Testing in this way gives you ultimate confidence that the code works!

To automate releases of new versions of terraform provider kong I have used the excellent goreleaser. Goreleaser automates building your code for multiple platforms, zipping/tar balling it up and uploading it to github. It is so easy to use, kudos to the authors. To setup you simply need to create a goreleaser.yml file:

binary: terraform-provider-kong
format: zip

view raw


hosted with ❤ by GitHub

My file specifies the binary and builds for mac (Darwin), Linux and Windows.  The last part is to define a task to run goreleaser as part of your build.  Goreleaser will only run when the build is running as a tag build, therefore you can create a task and run it on every build and goreleaser will only build a release for a tagged build, pretty smart.  My build task to create a release looks like:

go get github.com/goreleaser/goreleaser; \
goreleaser; \

view raw


hosted with ❤ by GitHub

I run the release task as part of every travis build.  You also need to create a github token which is used by goreleaser to upload the release to your github account.  To generate a github token go to https://github.com/settings/tokens/new.  Then simply set it as a secure environment variable in your .travis.yml and goreleaser will pick it up for you.

Then when you want to create a release you can simply do:

git tag -a v0.1.0 -m "My release"
git push origin v0.1.0

view raw

make release

hosted with ❤ by GitHub

Which will build a new v0.1.0 release and automatically create the binaries and upload the release to Github.  Pretty smart!

The last thing I wanted to go through was that I’m taking advantage of the build matrix feature of travis to run the tests against multiple versions of kong. The following section of my travis.yml file:


view raw


hosted with ❤ by GitHub

Travis will automatically run a build for each line under the matrix definition.  Each with the values on the line.  I pass the KONG_VERSION parameter all of the way through to the code that pulls the Kong container for Docker so it will pull the version that is specified here.  Hence I can easily update this build to run against many versions of Kong.  As new versions are released I can simply add a line to the build matrix and it will automatically be tested on travis.  This is a really powerful feature, being able to run your tests against multiple versions of a target piece of software is immense!

If you are interested terraform provider kong or gokong I would love to hear your feedback and get your ideas and contributions.



Full stack testing in golang with docker containers

I like to practice the approach of full stack component testing where the guiding principle is that you test the entire component from as high level as possible and only stub out third party dependencies (read other APIs) or something that isn’t easily available as a docker container.  I have recently started a golang project to write a client for kong and I thought this would be a good opportunity to use this testing strategy in a golang project.

I love Go but the one thing I don’t like so much about it is the approach that most people seem to be taking to testing. A lot of tests are at the method level where your tests end up being tightly coupled to your implementation. This is a bad thing. You know a test is not very good when if you have to change your test when you change your implementation. The reason this is bad is because firstly as you are changing your test the same time as you are changing your code once you have finished you have know way of knowing if the new implementation still works as the test has changed. It also restricts how much you can get in and edit the implementation as you have constantly having to update the way the tests mock out everything. By testing at the top component level the tests do not care about the implementation and the code runs with real components so it works how it will in the real world. By writing tests in this way I have seen a lot less defects and have never had to manually debug something.

Anyway back to the subject of the full stack testing approach in Go. To start I used the excellent dockertest project which gives you a great API to start and stop docker containers. I then took advantage of the fact that in a Go project there is a special test function that gets called for every test run:

func TestMain(m *testing.M) {
// setup
code := m.Run()
// teardown

view raw


hosted with ❤ by GitHub

In the above method you can do your test setup code where I have placed the //setup comment and your teardown code where I have placed the //teardown comment.  The code that gets returned by m.Run() is the exit code from the test run.  Go sets this to non zero if the test run fails so you need to exit with this code so your build will fail if your test run fails.  Now using this method I can start the kong docker container, run the tests and then stop the kong docker container.  Here is the full TestMain code at time of writing:y

func TestMain(m *testing.M) {
testContext := containers.StartKong(GetEnvVarOrDefault("KONG_VERSION", defaultKongVersion))
err := os.Setenv(EnvKongAdminHostAddress, testContext.KongHostAddress)
if err != nil {
log.Fatalf("Could not set kong host address env variable: %v", err)
code := m.Run()

view raw


hosted with ❤ by GitHub

I have wrapped the starting and stopping of the kong container in a method to abstract away the detail.  Notice how the StartKong method takes the Kong version as a parameter.  It gets the Kong version either from the environment variable KONG_VERSION or if that environment variable is not set then it uses the default Kong version which I set to the latest version 0.11 at time of writing.  The cool thing about this is that if I want to run my tests against a different version of Kong I can do that easily by changing this value.  The really cool thing about this is that I can run the build against multiple versions of Kong on travis-ci by taking advantage of the env matrix feature.  If you list multiple values for an environment variable in travis-ci then travis-ci will automatically run a build for each entry.  This means it is really easy to run the whole test pack against multiple versions of Kong which is pretty neat.  You can check out the gokong build to see this in action!

The one part you may be wondering from all of this is how do I get the url of the container that Kong is running on for use in my tests.  That is done by setting an environment variable KONG_ADMIN_ADDR.  The client uses that environment variable if set and if not then it defaults to localhost:8001.

With all of this in place it allows me to test the client by hitting a real running Kong in a container, no mocks in sight!  How cool is that.  Plus I can run against any version of Kong that is built as a docker container with a flick of a switch!

Here is an example of what a test looks like so you can get a feel:

func Test_ApisGetById(t *testing.T) {
apiRequest := &ApiRequest{
Name: "test-" + uuid.NewV4().String(),
Hosts: []string{"example.com"},
Uris: []string{"/example"},
Methods: []string{"GET", "POST"},
UpstreamUrl: "http://localhost:4140/testservice",
StripUri: true,
PreserveHost: true,
Retries: 3,
UpstreamConnectTimeout: 1000,
UpstreamSendTimeout: 2000,
UpstreamReadTimeout: 3000,
HttpsOnly: true,
HttpIfTerminated: true,
apiClient := NewClient(NewDefaultConfig()).Apis()
createdApi, err := apiClient.Create(apiRequest)
assert.Nil(t, err)
assert.NotNil(t, createdApi)
result, err := apiClient.GetById(createdApi.Id)
assert.Equal(t, createdApi, result)

view raw


hosted with ❤ by GitHub

I think that is really clean and readable.  All of the code that boots up and tears down Kong is out of sight and you can just concentrate on the test.  Again with no mocks around 🙂

If you want to see the rest of the code or help contribute to my gokong project that would be great.  I look forward to any feedback you have on this.

Creating a Team City 9 server on a Windows Azure VM

As part of the crane open source project that I talked about in this post, I am setting up a Team City Server in Azure.

I ran into a few problems so I wanted to document the process from start to finish here:


  1. Create a new VM on azure using the Azure management portal selecting a Windows Server 2012 R2 machine
  2. Click on “all items” on the left, select the new virtual machine then click ‘connect’ in the bar at the bottom.  This will download an rdp file configured to remote in to the desktop.
  3. Log in to the new vm using the username and password you setup in step 1
  4. At this point I wanted to download and install Team City but I had real trouble getting Team City to download in IE.  I then tried to download Chrome and I could not get Chrome to download either.  So I ended up installing chocolately and then installing Chrome through chocolatey.  If anyone reading this knows a better way please let me know.
  5. Open powershell as admin
  6. Run the command “Set-ExecutionPolicy Unrestricted”
  7. Run the command “iex ((new-object net.webclient).DownloadString(‘https://chocolatey.org/install.ps1‘))”
  8. Chocolately should now be installed.  Next we need to install Chrome using chocolatey.  Run the command “choco install GoogleChrome”
  9. Open up google chrome and download the Team City 9 EAP (or whichever version of Team City you want to run)
  10. Run the Team City installer, for ease I would use port 80 for the port.  Select the local system account both for the team city server and agent service
  11. Team City should now be up and running, feel free to configure your builds using crane 🙂
  12. Now at this point I thought (naively) that it would all just work but that’s not the case.  You need to setup some firewall rules to allow traffic through
  13. Open server manager (by default its the first item in the task bar with the picture of the suitcase)
  14. Click “Tools > Windows firewall with advanced security” to open up the firewall rules window
  15. Select “inbound rules” then “add new rule”, this will open up the new rule wizard.
  16. Select “port” as the rule type, click next.
  17. Type 80 in the port box (or whichever port you used for Team City), click next
  18. Click “allow the connection”, click next
  19. Leave the profile as is, click next
  20. Then click next on the last screen
  21. Now we have configured Windows Server to allow the connection, now we just have to open up the load balancer in azure…
  22. Go back to the azure management portal
  23. Click on all items on the left and select your virtual machine
  24. Select endpoints at the top
  25. Click add at the bottom
  26. Select add a “stand alone endpoint” click next
  27. If you have used port 80 then you can simply select “http” from the drop down list and click next.  If you have used a custom Team City port then you will need to set up the private port to the port you put Team City on and the public port to the port you want to use on the internet.  For example if you put Team City on port 8000 but you want to access it on port 80 you would select port 8000 for the private port and 80 for the public port.  Click next.
  28. Once Azure finishes updating you should now be able to access your Team City server on the internet by going to <vmname>.cloudapp.net

I hope you found this helpful and it has saved you some time.  I don’t think a lot of that is obvious out of the box.

Crane – Building you into the pit of success

I have recently started an open source project with a colleague (Ed Wilde). The project is called crane check it out on github. The idea behind the project is to write a tool that creates .Net builds for you automatically. For those of you who have spent time on a build server like Team City (not to pick on Team City as all build servers suffer from this) it takes quite a bit of time just to set up a boiler plate build.

A lot of that time is spent configuring the steps of the build normally something along the lines of the following:

  1. Get latest source
  2. Updated assembly build versions
  3. Build software
  4. Run tests
  5. Pack into nuget
  6. Publish into nuget

It can be confusing even for a developer with experience to set the build up in the right way and not only that but in a way that scales well as more applications are built.

This is where crane comes in.  The idea is you can point crane at your solution and it will generate you a build automatically with all of the steps in place.  By convention.  We believe we can use our experience of configuring builds to give you a great starting place either for a brand new project or to create a build for an existing project.

We are also planning to give crane the ability to create build chains by analysing your solutions.  This will give you the ability to split up your code base and work on smaller projects without the headache of trying to build it.

We would love to hear your initial thoughts on crane and whether it would be a useful tool for you.