The joy of LESS

I’ve been using LESS quite a lot of the last few years, especially combined with Bootstrap. Writing bog standard CSS feels quite long winded and painful now. Today I was reminded quite how much I like it when doing some categorisation for a site. You know the problem, lots of very similar lines of CSS with one differing attribute (in this case, colour). In standard CSS, you’d end up hard coding category names in multiple places, i.e:

.myDiv .my-category-one a, .myDiv .my-category-one h1 {color:red;}
.myDiv .my-category-two a, .myDiv .my-category-two h1  {color:blue;}
.myDiv .my-category-three a, .myDiv .my-category-three h1 {color:green;}

etc..

Behold, the mixin. Save yourself some time.

// Category Mixin
.category(@name, @colour){
  .myDiv [email protected]{name} {
    a {color: @colour}
    h1 {color: @colour}
  }
} 
// Create Categories
.category(my-category-one, red);
.category(my-category-two, blue);
.category(my-category-three, green);

Why is this better? Well, to add another Category is simply a case of adding

.category(my-next-category, @brand-primary);

Anything from my Mixin, like H1 and a CSS selectors get auto applied. Make the change once, done. Need to add more selectors to your category? Add them to the mixin. Simples. Note the variable output of “name” – [email protected]{name} which will create .my-category-one (or whatever you’ve passed in).

Setting up HTTPS on Elastic Beanstalk

Edit: AWS now have a certificate service which may be more useful to you, depending on your setup.

The fundamental difference in setting up https on Elastic Beanstalk is that your certificate is installed on the load balancer/environment itself, *not* the individual instances. This actually makes life a lot easier: you don’t need to worry about doing anything to the instances themselves as the SSL terminates at the load balancer. This assumes you’re using multiple availability zones, not a single zoned instance.

Also, remember this won’t ever work on myenvironment.beanstalk.com, only on a CNAME entry for yourdomain.com which you’ve aliased to the beanstalk environment URL, as obviously, AWS don’t hand out certs for beanstalk.com 🙂

Step 1:
Get your certificates and ensure they’re in the right format.

I used sslmate.com, which I found very easy to use for a simple DV (Domain validated) certificate – you don’t actually need *more* than DV unless you want to do things like have “yourcompany” in the URL bar alongside the padlock sign. Wildcards are only really cost effective if you want more than 8-9 subdomains certified.

So once sslmate is installed locally, do:

$ sslmate buy www.example.com

and follow the prompts. Note, I didn’t add any of the ‘optional’ fields as I remember that can cause problems (unfortunately I can’t remember exactly *where* I read that).

Once you’ve successfully made the purchase, you should now have a bunch of files in the directory you ran the command. The good part is these certificates are in the correct format, and also are signed. The PEM format (as opposed to DER) is actually not necessarily to do with the file extension: i.e, .crt can be in PEM format, so don’t feel you need to convert these certificates just because they haven’t got a .pem extension. I only mention formats as this is the ‘other’ format which AWS uses for S3 key pair URL signing, so if you’re only exposure to that sort of thing is S3, this is the other one.

You should have:

mydomain.com.key
mydomain.com.crt
mydomain.com.chain.crt
mydomain.com.chained.crt

Step 2:

Make sure you’ve got AWS command line tools installed. This isn’t the Elastic Beanstalk CLI tools – this is in addition to those. You can get them here: http://aws.amazon.com/cli/ – although as I had pip installed from installed the EB CLI, I just did

$ pip install awscli

After which, setup using:

$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-west-1
Default output format [None]: json

Step 3:

Time to upload our certs to a AWS IAM profile:

$ aws iam upload-server-certificate --server-certificate-name certificate_object_name --certificate-body file://public_key_certificate_file --private-key file://privatekey.pem --certificate-chain file://certificate_chain_file

So this should equate to something like:

$ aws iam upload-server-certificate --server-certificate-name myDomainSSL --certificate-body file://mydomain.com.crt --private-key file://mydomain.com.key --certificate-chain file://mydomain.com.chain.crt

The Chain file is important for browser acceptance.

If successful, you should get something like:

arn:aws:iam::123456789012:server-certificate/cert

returned at the prompt.

Step 4:

Attach cert to the load balancer:

Login to the web GUI at aws.amazon.com, select Elastic Beanstalk and click on the configuration section for the environment you want to apply the certificate to. Under Network Tier > Load Balancing, click on the cog to configure.

You should be able to see “myDomainSSL” as a dropdown under SSL Certificate ID – if you don’t, the certificate isn’t in IAM or something so you need to go back a few steps and check.

So at this point, you need to do something like:

Listener Port: 80
Protocol: HTTP
Secure Listener Port: 443
Protocol: HTTPS

and ensure your certificate is selected.

Saving will restart your instance, and all going well, https://www.mydomain.com should work. Except if you’re using VPC, you need to do one last step, which is open up 443 on the security group; (Go to EC -> Security Groups, select Load Balancer Security Group, add an inbound rule for HTTPS/TCP/443 on 0.0.0.0/0)

Step 5:

Read something like This for some nice pointers/reminders about running an app over https.

Railo on Elastic Beanstalk, finally.

Also known as “A Crawl through Amazon Web Services.”

So over the last few weeks I’ve been experimenting properly with Elastic Beanstalk as a home for an app which needs some serious resiliency; it’s not a high network/throughput app, but each request is critical. It can’t afford to have 5xx server errors etc. and network timeouts would be a disaster.

So naturally, I thought one method to try would be a load-balanced Elastic Beanstalk setup, with different instances in different availability zones.

Moving an app from a VPS to “the cloud” isn’t always straightforward it seems – there are some fundamental shifts in mentality which can lead to a puzzling combination of frustration and eureka moments. There’s *some* documentation out there specifically for Railo, a couple of good blog posts, but I found there was always something which wasn’t working, from the setup being on older version of Eclipse, to old documentation for the EB Command Line interface.

So here, in the first week of 2015, I’ll try and outline what appears to be working for *me* at *this given time*.

So, lets try to get a cfWheels App running off Amazon RDS + Elastic Beanstalk using the Tomcat 8 container….

Usual disclaimers apply. Now, some additional disclaimers.

  1. I don’t like Eclipse, and I don’t want to use it: it’s never really worked for me, from installing CF extensions, to installing the AWS CLI, to launching Tomcat instances. So what follows below is trying to only use a text editor (sublime in my case), the AWS web console, commandBox and the Elastic Beanstalk (henceforce abbreviated to “EB”) Command Line Interface (CLI).
  2. I use both Windows and Mac. This setup is easier on Mac, but it does work on Windows.
  3. I tried using Docker via Adam Chapmans Docker-Railo repo ; it would deploy in a fresh EB environment perfectly, but any future updates just seemed to kill the docker process. I know Adam does this via Jenkins, so I think he has better luck that way. That said, there’s some nice bits in there, with nginx and cfWheels url rewriting out of the box, which is awesome.

So, moving on. Things you will need:

  1. A working AWS account, including access + secret keys etc. I’m going to go ahead and assume you’ve got that.
  2. You’ll want Python installed on your dev machine. 2.7 or later (3.4 on Windows).
  3. Then you’ll want ‘pip
  4. A working install of the Amazon EB CLI. (version 3.x) (installed via pip)
  5. Git. (I use mysysgit on windows)
  6. Commandbox (this isn’t 100% essential, but you’ll see why)
  7. Railo “war” style. (download the “WAR Archive” under “All OS”)

Step 0: get your Tools

If you’ve got all these working properly, you should be able to do the following commands without throwing errors:

$ git --version
$ python --version
$ pip --version
$ eb --version
$ box version

Got that? Good. With mac, these are all easily accessed with terminal. With windows, you’ll need to set some $PATH variables if you want to get everything working out of a single prompt. I tend to leave commandbox in it’s own one, and try and get everything running through cmd.exe at a minimum, but in an ideal world, you’ll want powershell or something of that ilk. Not 100% necessary though.

Step 1: get cfWheels

Open commandbox (I’m going to assume all commandbox stuff is running in the box.exe shell and not your OS native shell), run:

$ mkdir myApp
$ cd myApp
$ install cfwheels

Move the entire contents of the cfwheels folder into the root of the project folder “myApp” and delete the cfwheels folder. (Alternatively, just download cfwheels and chuck it in a folder, your call).

Step 2: make it a Git repo

Then, we want to make this a git repository. This is essential. We will be using git branches to associate with EB environments. i.e, we might have a “staging” branch which we will run independently to “production”.

So let’s establish the repo:

$ git init

You should get “initialised empty git repository” as a response.

We’re also going to need some branches – so let’s create a staging branch:

$ git checkout -b staging

Next, we’re going to chuck in Railo. Go to your downloaded railo WAR file, and rename it to .zip. Decompress the zip, and pull out the whole WEB-INF folder. Add to the myApp folder.

Before we do anything else, we need to get some .gitignore details in. Open up .gitignore (or create it if it doesn’t exist), and add the contents of https://gist.github.com/neokoenig/200ad6b9b38c5bb27e9a

Most of that should look pretty standard, but you’ll see a load of WEB-INF/railo/etc entries. More on that later.

Now we’ve got a sensible base from which to work, let’s commit this.

$ git add -A
$ git commit -am "Add Wheels and Railo"

If we want to be 100% sure we’ve got that all working, then this:

$ git status

Should return “On branch Staging, nothing to commit, working directory clean”.

So we’ve got a blank cfWheels install, and it’s in a git repo.

Step 3: RDS = Data, data and more data.

One of the fundamental changes in mindset from moving to a VPS is the requirement to completely isolate your data from your application. There’s your usual database rows, i.e models etc, but we also need to setup a database for Railo’s sessions. Due to the way EB works, when you start or destroy instances due to autoscaling, you can’t rely on the application session scope, as under load, you may be logged into one instance, but then the load balancer will throw you to a new instance, where your session doesn’t exist. Pretty obvious when you think about it.

Thankfully Railo has a nice simple way of setting up sessions to run from a datasource, which we’ll come to later. For now, I want to setup a RDS instance, and add three databases, we’ll call them myApp_production, myApp_staging and myApp_sessions. You’ll be able to setup one on creation, but you’ll need to login separately to add the other two later.

I’m not going to go into the finer points of setting up a RDS instance, as it’s pretty straight forward. I went with the bog standard mySQL, Multi A-Z deployment, t2.micro setup. Once you’re done, you should have a master username & password, and an most importantly, an endpoint, which will look something like:

myApp_sessions.casdl234nwb1vur.eu-west-1.rds.amazonaws.com:3306

This is (minus the :3306) your database hostname. I tend to access mySQL via Navicat, so once you’ve added your local IP to the security group (go to EC2 -> security Groups), you should be able to access the RDS instance using the credentials you just got. Add the other two databases if you’ve not done already.

At this point, you’ll probably want to add the default datasources in cfWheels – as we’ve bothered setting up a development/staging database, I usually do something like:

if(cgi.SERVER_NAME CONTAINS "www.mydomain.com"){
set(dataSourceName="myApp_production");
} else {
set(dataSourceName="myApp_staging");
}

But again, how you switch datasources is up to you – some people like setting them in config/development/settings.cfm and config/production/settings.cfm respectively, but I prefer mainly running in production mode, even on my staging environment.

Step 4: Railo configuration

So we’ve got railo sitting in our /WEB-INF/ directory. At the moment, you’ll just have a /lib/ folder and web.xml file. That’s fine. What we want to do now is start adding some configuration to Railo locally, which will then get added to our git Repo, and subsequently deployed to AWS.

There are two distinct ways of doing this. One is to run Railo locally on an app server, enter your datasources etc, and the other is to add the exported details directly to the /config/app.cfm file in cfwheels so that you get the application overriding the default behaviour. Since we’ve got to do things like set the Railo administrator password, I’d probably just stick with doing everything the first way, but it’s useful to know that you can set datasources, defaults for cfmail tags etc directly within Application.cfc.

So let’s boot Railo locally, using commandbox.

In commandbox, navigate to your myApp directory, and run:

$ server start

You should get a “server starting on port XXXXX”, and your browser should open. Hopefully you’ll see the default “Welcome to Wheels” page too. Go to /railo-context/admin/server.cfm : on first entry we need to set a admin password (you’ll need to do this on the web.cfm context too!). I’m going to assume you know how to add datasources to Railo: just use the RDS connection strings from earlier, so we can setup myApp_production and myApp_staging as normal datasource names.

The sessions database bit requires a different setup – instructions here .

Other bits to remember to change are the mail server settings (I use mandrill SMTP service – highly recommended), changing the error handling templates to ones which don’t give tonnes of information away, maybe the timezone information, and changing the session scope to the myApp_sessions datasourcename.

We can probably shut down commandbox’s server at this point:

$ server stop

We now need to add that configuration which we’ve done in the railo admin to the git repo.

$ git add -A
$ git commit -am "Add Railo Config"

As we bothered to setup the .gitignore earlier, hopefully files specific to the local installation (except the configuration files etc) should be ignored. If you want to check it’s definitely worked, then open up /WEB-INF/railo/railo-web.xml.cfm and you should see the encrypted password near the top.

Step 5: Elastic Beanstalk

We’re finally at a stage where we can consider deploying to Elastic Beanstalk.

So let’s first configure the eb environment:

$eb init

Select a default region

1) us-east-1 : US East (N. Virginia)
2) us-west-1 : US West (N. California)
3) us-west-2 : US West (Oregon)
4) eu-west-1 : EU (Ireland)
5) eu-central-1 : EU (Frankfurt)
6) ap-southeast-1 : Asia Pacific (Singapore)
7) ap-southeast-2 : Asia Pacific (Sydney)
8) ap-northeast-1 : Asia Pacific (Tokyo)
9) sa-east-1 : South America (Sao Paulo)
(default is 3): 4

Select an application to use
1) adifferentApplication
2) [ Create new Application ]
(default is 2): 2

Enter Application Name
(default is “myApp”):
Application myApp has been created.

It appears you are using Tomcat. Is this correct?
(y/n): y

Select a platform version.
1) Tomcat 8 Java 8
2) Tomcat 7 Java 7
3) Tomcat 7 Java 6
4) Tomcat 7
5) Tomcat 6
(default is 1): 1

Do you want to set up SSH for your instances?
(y/n): n

If you login to AWS now, you’ll see your new application under Elastic Beanstalk, but with no environments.

So let’s make one:

$ eb create staging

This will take a while: it may even look like the command prompt has hung as you don’t get an immediate response. Don’t panic, let it be. It’s packaging up everything you’ve done to this point and sending it Elastic beanstalk. Whilst you’re waiting, check your .elasticbeanstalk/config.yml. It should look something like this:

branch-defaults:
staging:
environment: null
global:
application_name: myApp
default_ec2_keyname: null
default_platform: Tomcat 8 Java 8
default_region: eu-west-1
profile: eb-cli
sc: git

So if you need to do anything really drastic in the future, you can just alter bits here if you want.

Whilst we set this up via CLI, you can easily do this from the web GUI if that makes you happier; you just need to specify the application name when doing eb init later.

After a while you should start to see some info coming in from EB. The first time you do this it has to create all the auto-scaling groups, cloudwatch alarms, security groups etc.

When it’s all done, *don’t* visit your new environment URL just yet. There’s one more step – we need to add our new environment to the security group for the RDS instance, otherwise we’ll get a very unhelpful WSOD (white screen of death) as the page will timeout trying to connect to RDS to query for your session! So if you remember from before, we need to go into EC2-> Security groups, and add the environment security group to the RDS mySQL group: exactly like you did when you wanted to connect from it locally, except this time, we’re going to do “custom IP” and then start typing the ‘sg…” until your environment security group pops up.

Aaaand, that’s it. Go, visit your lovely cfWheels welcome page.

Updating your App:

Simply make your changes, commit to git, then:

$ eb deploy

Don’t worry, it doesn’t take quite as long to update (but it can take a while). It does update via deltas (I believe) so it can be pretty quick – you may want to think about having 2+ instances and doing a rolling update if you want to avoid any downtime at all.

Step 7: Add Tuckey URL rewrite for cfwheels:

If you’ve followed the above steps, adding url rewriting is pretty easy.

Download and install the .jar file – instructions here: http://tuckey.org/urlrewrite/

Then add this example set of rules: https://gist.github.com/neokoenig/9d467c585f90292a6a8a

Then git commit, and eb deploy. Note how I put health.html as an exception condition here (see below).

Adding a Production branch:

Is as simple as creating a new branch like before, but doing “production” and then creating a production environment. As your logic to switch datasources is in the app itself, you can just merge staging into production and deploy really quickly. Remember to add any new environments to your RDS security group.

Other notes:

One of the issues I hit was the EB Health monitoring, with requests from Tomcat not containing the correct headers to satisfy the monitoring script. This meant an otherwise working environment would register as “Red”, when actually it was fine. To help with this, I just created an html file in the root called ‘health.html’ which just contains minimal HTML markup, and pointed the health check to /health.html. YMMV.

Remember, sessions from a database will incur more database requests – obvious huh, but when you’ve got 80 concurrent HTTP requests requesting at least one normal DB call, that’s 160 requests to RDS. You may want to consider whether you app actually needs a session scope at all, or the Ben Nadel trick of setting an ultra short session time (until you properly login) so that server memory gets saved, and that Railo then doesn’t do the ‘dump session in database after 10 seconds’ thing.

If you want to upgrade your instance’s from t1.micro -> t2.micro, you can’t currently do this: you have to create a new environment and then switch environment URLS. So ideally, try and select the appropriate machine size to start with.

Autoscaling is great, but… don’t just throw 2500 requests a second at your environment using loader.io or something. Scaling has triggers, such as network output is over ‘x’ for ‘x’ minutes, so it takes a while to warm up. If you simply shove a tonne of concurrent requests at an instance without setting a minimum number of instances of more than one, or prewarming it somehow, you’ll hit memory limits and it’ll fall over, depending on your machine type. Consider different autoscaling triggers (page latency is a fairly good one) other than network output too. And don’t forget to set the scale-in triggers as well as scale-out. You don’t want to trigger 20 machines and for them to never go back to 1-2.