Hooking Bitbucket up with Jenkins parameterized jobs

Bitbucket repositories allow us to setup hooks which notify/trigger Jenkins' jobs about newly pushed code. The process to create such a hook is documented here. However, it doesn't mention how to integrate with Jenkins parameterized jobs.
After reading the Jenkins documentation and a few trial and error I managed integrate Bitbucket and Jenkins parameterized jobs.
The hook management form presents 4 fields:

1. Endpoint: Here, you’ll need to set your Jenkins URL in the following format — http://username:apitoken@your.jenkins.url/job/your.job.name/buildWithParameters 
2. Module name: (Optional)
3. Project name: (Leave Empty)
4. Token: It’s the authentication token you defined in your Jenkins job settings 

The gotcha is leaving the project name field blank and include it in the endpoint URL appended with buildWithParameters.

Setting up Jenkins on EC2 using AWS CloudFormation (including nginx as a reverse proxy)

I lost count on how many times I had to setup a CI server. Being a big fan of the concept of "Infrastructure as Code" as I am, I promised myself that the next time I'd do it differently (I needed to have it automated!).

Well, the day has come.The stack (using Cloudformation terms) I decided to create is composed by a basic Jenkins installation (standalone + winstone) running as a daemon on an Amazon Linux based EC2 intance. It also includes Nginx as a reverse proxy and a dedicated volume for storing the JENKINS_HOME files and other artifacts.

Obviously that a much easier, simpler and (I think) even cheaper alternative would be using some PaaS offering (Jenkins as a Service).

However, I ended up using AWS Cloudformation to automate the provisioning of the AWS Resources (IAM User, SecurityGroup, EBS Volumes, EC2 instance) and its cloud-init features to install and configure the necessary packages.

External repository addition, yum based package installations, configuration files adjustments, EBS volumes setup, etc. Most of the installation and configuration logic is in a shell script embedded in the UserData property.

The resulting template is right below, the input parameters are: Instance type, JENKINS_HOME volume size in gigabytes and the EC2 KeyName. The output is the Jenkins server URL!

Loading ....

Automating GAE's application deployment with Jenkins

Follow below the expect script I created to automate the deployment of applications on GAE. It simply invokes/spawns the appcfg.sh and sends the account's password when prompted.
Then I configured a Jenkins job which contains a "execute shell" step invoking this script.

Jenkins "execute shell" step

expect $WORKSPACE/appcfg.exp "$APPENGINE_HOME" "myuser@gmail.com" "mypassword" "update" "$WORKSPACE/war"

appcfg.exp expect script

#!/usr/bin/expect -f
# Expect script to supply GAE's account password for appcfg.sh
#
# This script needs four arguments:
# username = GAE's google account email
# password = GAE's google account password
# warDir = war directory to deploy to GAE
# gaeHome = GAE's SDK home dir
#
# For example:
#  expect appcfg.exp myemail@gmail.com mypassword ./war /usr/share/appengine-sdk-1.5.3

if {[llength $argv] == 0} {
   puts "usage: appcfg.exp {-index|#}"
   exit 1
}

set gaeHome [lrange $argv 0 0]
set username [lrange $argv 1 1]
set password [lrange $argv 2 2]
set cmd [lrange $argv 3 3]
set warDir [lrange $argv 4 4]

set timeout -1

# spawns appcfg.sh
spawn $gaeHome/bin/appcfg.sh --enable_jar_splitting --passin --email=$username $cmd $warDir
match_max 100000

expect {
   default {exit 0}
   # Look for passwod prompt
   "*?assword*"
}

# Send password aka $password
send -- "$password\r"

# send blank line (\r) to make sure we get back to gui
send -- "\r"
expect eof

Using XmlSlurper to update appengine-web.xml

I'm currently working on a project which is using GAE (Google Application Engine).
I wanted to setup Jenkins' jobs to update the different stages (acceptance, qa, uat) in our build pipeline.
To achieve that I created 3 different GAE's applications - one for each stage - and 4 Jenkins jobs: 1 template which holds the common steps and 3 others for the respective stages.
The first step of the template job updates appengine-web.xml target application and version based on parameters passed to the job.


See the groovy script below:

import groovy.xml.StreamingMarkupBuilder

//getting parameters values from environment variables
def env = System.getenv()
def workspace = env["WORKSPACE"]
def applicationName = env["TARGET_APPLICATION_NAME"]
def versionName = env["TARGET_VERSION_NAME"]
def ant = new AntBuilder()
ant.echo(message:"Opening $workspace/war/WEB-INF/appengine-web.xml")
def file = new File("$workspace/war/WEB-INF/appengine-web.xml")
def root = new XmlSlurper().parse(file)
ant.echo(message:"Updating appengine-web.xml with application: $applicationName and version: $versionName")
root.application=applicationName
root.version=versionName 
def outputBuilder = new StreamingMarkupBuilder()
String result = outputBuilder.bind{ 
   mkp.declareNamespace("":  "http://appengine.google.com/ns/1.0")    
   mkp.yield root 
}
ant.echo(message:"Writing appengine-web.xml")
file.write(result)