Continuous Deployment from Github to App Engine Java with CodeShip

Motivation

Continuous Deployment is nice: you roll out new versions of your software all the time and not only three, four times a year with your mayor release. And everything automatically. Of course this requires a good coverage of automated unit, integration and UI tests to make sure that your current version doesn’t break anything for the user.
What tools can you use do achieve this for your open source project? If I’m not mistaken you get all this with an account at bit bucket from Atlassian. But since I currently host my open source projects on Github I had to find another solution. And it exists, even for free: http://codeship.io
Codeship already nows about many services and you only have to configure a few lines to get it up and running. For instance it supports Github as code repository and Google App Engine as deployment target. But the documentation about it is still a bit sparse. So this article describes how you can set it up

SCM

The setup of github as a repository is fairly straight forward: you use your github account to authenticate yourself through OAuth with CodeShip. Then you create a new project and in the SCM section you pick Github as your provider. After allowing codeship to access your Github repositories you already can select the repo you want to build from.

Tests

Currently I don’t have integration and UI tests yet for my open source project. Anyway, to make sure your build runs at least once I chose the JVM option in the “Select your technology to prepopulate basic commands” dropdown and added a

mvn package

in the “Modify your Setup Commands” section. I left the rest unchanged.

Deployment

And now came the tricky part. I don’t want the build to deploy to the GAE application that is defined in the appengine-web.xml. Instead I want it to be installed to a test environment with a different name running on GAE as well. So I added a second appengine-web-testenv.xml beside the default one and changed the application id to the test environment. Then I selected the “$script” option in the CodeShip deployment tab and added a

cp target/-/WEB-INF/appengine-web-testenv.xml target/-/WEB-INF/appengine-web.xml

Please replace maven.project and maven.version with the name of your maven artefact. In my case this is “extraleague-1.0-SNAPSHOT”. So this basically replaces the original appengine file with the one containing the test environment id. When appcfg.sh now tries to find the target for deployment it will find the one with the test environment.
Now we add another deployment descriptor by selecting the app engine option. I think when you first do this you have to authorise CodeShip to deploy to GAE on your behalf. After that I added./target/extraleague-1.0-SNAPSHOT/ as Path and a rest service endpoint in the URL field. This URL will be checked to decide if the deployment was successful. In my first attempt I used the index.html of my angular application for this. The problem here is that static files like the index.html did get served while the Java server part had a 500 error. That’s why I think a rest point is a better choice here.

Triggering a run and debugging

To have CodeShip do a first run you just have to commit a change to Github and there it goes. You can follow each step in the admin interface of your build and see read the log if something goes wrong. You can also change the deployment settings and rerun a previously failed build.

Remarks

I’m really happy with this setup, but I think there should be a way to avoid setting the artefact name and version into the setting path. Probably some maven command would do the trick? As next steps I’d like to setup a UI test, maybe with PhantomJS or so… Let me know if you have improvements!

Update 2014/05/10
Last week I had issues with app engine authentication and after that was solved it seemed that codeship used an old version of the app engine sdk to deploy which caused issues. So I started looking for an alternative to the above description and found it by adding the following script in the deployment section (no other build step required anymore, you can remove the app engine step):

cp src/main/webapp/WEB-INF/appengine-web-testenv.xml  src/main/webapp/WEB-INF/appengine-web.xml
cat src/main/webapp/WEB-INF/appengine-web.xml
echo '{"credentials":{"rof":{"access_token":"XXXX","expiration_time_millis":1399667604244,"refresh_token":"XXX"}}}' > ~/.appcfg_oauth2_tokens_java
mvn appengine:update
wget --retry-connrefused --no-check-certificate -T 60 http://your-test-url

Explanation
The first line is the same as before: it copies my alternative deployment descriptor in the place of the default descriptor. The second line is used for debugging and to be sure that the task will really deploy to my test instance. The third line is the most interesting. For my own security I replaced the real tokens with XXX. How do you get this JSON object? Very simple: deploy your application locally and you can copy this JSON object from .appcfg_oauth2_tokens_java. Replace your current system user with rof! The next line simply deploys the application to my test environment and the last line checks if the application was deployed successfully