cfWheels Plugin Survey

Use cfWheels? Help us get an idea of what plugins you guys use, and which you think might qualify for some core team support! The survey will only take a minute – please do fill it in if you use any plugins in cfWheels..

https://www.surveymonkey.com/s/9P7PJCV

Roombooking System 1.2 Development Preview

Railo on Elastic Beanstalk, finally.

Also known as “A Crawl through Amazon Web Services.”

So over the last few weeks I’ve been experimenting properly with Elastic Beanstalk as a home for an app which needs some serious resiliency; it’s not a high network/throughput app, but each request is critical. It can’t afford to have 5xx server errors etc. and network timeouts would be a disaster.

So naturally, I thought one method to try would be a load-balanced Elastic Beanstalk setup, with different instances in different availability zones.

Moving an app from a VPS to “the cloud” isn’t always straightforward it seems – there are some fundamental shifts in mentality which can lead to a puzzling combination of frustration and eureka moments. There’s *some* documentation out there specifically for Railo, a couple of good blog posts, but I found there was always something which wasn’t working, from the setup being on older version of Eclipse, to old documentation for the EB Command Line interface.

So here, in the first week of 2015, I’ll try and outline what appears to be working for *me* at *this given time*.

So, lets try to get a cfWheels App running off Amazon RDS + Elastic Beanstalk using the Tomcat 8 container….

Usual disclaimers apply. Now, some additional disclaimers.

  1. I don’t like Eclipse, and I don’t want to use it: it’s never really worked for me, from installing CF extensions, to installing the AWS CLI, to launching Tomcat instances. So what follows below is trying to only use a text editor (sublime in my case), the AWS web console, commandBox and the Elastic Beanstalk (henceforce abbreviated to “EB”) Command Line Interface (CLI).
  2. I use both Windows and Mac. This setup is easier on Mac, but it does work on Windows.
  3. I tried using Docker via Adam Chapmans Docker-Railo repo ; it would deploy in a fresh EB environment perfectly, but any future updates just seemed to kill the docker process. I know Adam does this via Jenkins, so I think he has better luck that way. That said, there’s some nice bits in there, with nginx and cfWheels url rewriting out of the box, which is awesome.

So, moving on. Things you will need:

  1. A working AWS account, including access + secret keys etc. I’m going to go ahead and assume you’ve got that.
  2. You’ll want Python installed on your dev machine. 2.7 or later (3.4 on Windows).
  3. Then you’ll want ‘pip
  4. A working install of the Amazon EB CLI. (version 3.x) (installed via pip)
  5. Git. (I use mysysgit on windows)
  6. Commandbox (this isn’t 100% essential, but you’ll see why)
  7. Railo “war” style. (download the “WAR Archive” under “All OS”)

Step 0: get your Tools

If you’ve got all these working properly, you should be able to do the following commands without throwing errors:

$ git --version
$ python --version
$ pip --version
$ eb --version
$ box version

Got that? Good. With mac, these are all easily accessed with terminal. With windows, you’ll need to set some $PATH variables if you want to get everything working out of a single prompt. I tend to leave commandbox in it’s own one, and try and get everything running through cmd.exe at a minimum, but in an ideal world, you’ll want powershell or something of that ilk. Not 100% necessary though.

Step 1: get cfWheels

Open commandbox (I’m going to assume all commandbox stuff is running in the box.exe shell and not your OS native shell), run:

$ mkdir myApp
$ cd myApp
$ install cfwheels

Move the entire contents of the cfwheels folder into the root of the project folder “myApp” and delete the cfwheels folder. (Alternatively, just download cfwheels and chuck it in a folder, your call).

Step 2: make it a Git repo

Then, we want to make this a git repository. This is essential. We will be using git branches to associate with EB environments. i.e, we might have a “staging” branch which we will run independently to “production”.

So let’s establish the repo:

$ git init

You should get “initialised empty git repository” as a response.

We’re also going to need some branches – so let’s create a staging branch:

$ git checkout -b staging

Next, we’re going to chuck in Railo. Go to your downloaded railo WAR file, and rename it to .zip. Decompress the zip, and pull out the whole WEB-INF folder. Add to the myApp folder.

Before we do anything else, we need to get some .gitignore details in. Open up .gitignore (or create it if it doesn’t exist), and add the contents of https://gist.github.com/neokoenig/200ad6b9b38c5bb27e9a

Most of that should look pretty standard, but you’ll see a load of WEB-INF/railo/etc entries. More on that later.

Now we’ve got a sensible base from which to work, let’s commit this.

$ git add -A
$ git commit -am "Add Wheels and Railo"

If we want to be 100% sure we’ve got that all working, then this:

$ git status

Should return “On branch Staging, nothing to commit, working directory clean”.

So we’ve got a blank cfWheels install, and it’s in a git repo.

Step 3: RDS = Data, data and more data.

One of the fundamental changes in mindset from moving to a VPS is the requirement to completely isolate your data from your application. There’s your usual database rows, i.e models etc, but we also need to setup a database for Railo’s sessions. Due to the way EB works, when you start or destroy instances due to autoscaling, you can’t rely on the application session scope, as under load, you may be logged into one instance, but then the load balancer will throw you to a new instance, where your session doesn’t exist. Pretty obvious when you think about it.

Thankfully Railo has a nice simple way of setting up sessions to run from a datasource, which we’ll come to later. For now, I want to setup a RDS instance, and add three databases, we’ll call them myApp_production, myApp_staging and myApp_sessions. You’ll be able to setup one on creation, but you’ll need to login separately to add the other two later.

I’m not going to go into the finer points of setting up a RDS instance, as it’s pretty straight forward. I went with the bog standard mySQL, Multi A-Z deployment, t2.micro setup. Once you’re done, you should have a master username & password, and an most importantly, an endpoint, which will look something like:

myApp_sessions.casdl234nwb1vur.eu-west-1.rds.amazonaws.com:3306

This is (minus the :3306) your database hostname. I tend to access mySQL via Navicat, so once you’ve added your local IP to the security group (go to EC2 -> security Groups), you should be able to access the RDS instance using the credentials you just got. Add the other two databases if you’ve not done already.

At this point, you’ll probably want to add the default datasources in cfWheels – as we’ve bothered setting up a development/staging database, I usually do something like:

if(cgi.SERVER_NAME CONTAINS "www.mydomain.com"){
set(dataSourceName="myApp_production");
} else {
set(dataSourceName="myApp_staging");
}

But again, how you switch datasources is up to you – some people like setting them in config/development/settings.cfm and config/production/settings.cfm respectively, but I prefer mainly running in production mode, even on my staging environment.

Step 4: Railo configuration

So we’ve got railo sitting in our /WEB-INF/ directory. At the moment, you’ll just have a /lib/ folder and web.xml file. That’s fine. What we want to do now is start adding some configuration to Railo locally, which will then get added to our git Repo, and subsequently deployed to AWS.

There are two distinct ways of doing this. One is to run Railo locally on an app server, enter your datasources etc, and the other is to add the exported details directly to the /config/app.cfm file in cfwheels so that you get the application overriding the default behaviour. Since we’ve got to do things like set the Railo administrator password, I’d probably just stick with doing everything the first way, but it’s useful to know that you can set datasources, defaults for cfmail tags etc directly within Application.cfc.

So let’s boot Railo locally, using commandbox.

In commandbox, navigate to your myApp directory, and run:

$ server start

You should get a “server starting on port XXXXX”, and your browser should open. Hopefully you’ll see the default “Welcome to Wheels” page too. Go to /railo-context/admin/server.cfm : on first entry we need to set a admin password (you’ll need to do this on the web.cfm context too!). I’m going to assume you know how to add datasources to Railo: just use the RDS connection strings from earlier, so we can setup myApp_production and myApp_staging as normal datasource names.

The sessions database bit requires a different setup – instructions here .

Other bits to remember to change are the mail server settings (I use mandrill SMTP service – highly recommended), changing the error handling templates to ones which don’t give tonnes of information away, maybe the timezone information, and changing the session scope to the myApp_sessions datasourcename.

We can probably shut down commandbox’s server at this point:

$ server stop

We now need to add that configuration which we’ve done in the railo admin to the git repo.

$ git add -A
$ git commit -am "Add Railo Config"

As we bothered to setup the .gitignore earlier, hopefully files specific to the local installation (except the configuration files etc) should be ignored. If you want to check it’s definitely worked, then open up /WEB-INF/railo/railo-web.xml.cfm and you should see the encrypted password near the top.

Step 5: Elastic Beanstalk

We’re finally at a stage where we can consider deploying to Elastic Beanstalk.

So let’s first configure the eb environment:

$eb init

Select a default region

1) us-east-1 : US East (N. Virginia)
2) us-west-1 : US West (N. California)
3) us-west-2 : US West (Oregon)
4) eu-west-1 : EU (Ireland)
5) eu-central-1 : EU (Frankfurt)
6) ap-southeast-1 : Asia Pacific (Singapore)
7) ap-southeast-2 : Asia Pacific (Sydney)
8) ap-northeast-1 : Asia Pacific (Tokyo)
9) sa-east-1 : South America (Sao Paulo)
(default is 3): 4

Select an application to use
1) adifferentApplication
2) [ Create new Application ]
(default is 2): 2

Enter Application Name
(default is “myApp”):
Application myApp has been created.

It appears you are using Tomcat. Is this correct?
(y/n): y

Select a platform version.
1) Tomcat 8 Java 8
2) Tomcat 7 Java 7
3) Tomcat 7 Java 6
4) Tomcat 7
5) Tomcat 6
(default is 1): 1

Do you want to set up SSH for your instances?
(y/n): n

If you login to AWS now, you’ll see your new application under Elastic Beanstalk, but with no environments.

So let’s make one:

$ eb create staging

This will take a while: it may even look like the command prompt has hung as you don’t get an immediate response. Don’t panic, let it be. It’s packaging up everything you’ve done to this point and sending it Elastic beanstalk. Whilst you’re waiting, check your .elasticbeanstalk/config.yml. It should look something like this:

branch-defaults:
staging:
environment: null
global:
application_name: myApp
default_ec2_keyname: null
default_platform: Tomcat 8 Java 8
default_region: eu-west-1
profile: eb-cli
sc: git

So if you need to do anything really drastic in the future, you can just alter bits here if you want.

Whilst we set this up via CLI, you can easily do this from the web GUI if that makes you happier; you just need to specify the application name when doing eb init later.

After a while you should start to see some info coming in from EB. The first time you do this it has to create all the auto-scaling groups, cloudwatch alarms, security groups etc.

When it’s all done, *don’t* visit your new environment URL just yet. There’s one more step – we need to add our new environment to the security group for the RDS instance, otherwise we’ll get a very unhelpful WSOD (white screen of death) as the page will timeout trying to connect to RDS to query for your session! So if you remember from before, we need to go into EC2-> Security groups, and add the environment security group to the RDS mySQL group: exactly like you did when you wanted to connect from it locally, except this time, we’re going to do “custom IP” and then start typing the ‘sg…” until your environment security group pops up.

Aaaand, that’s it. Go, visit your lovely cfWheels welcome page.

Updating your App:

Simply make your changes, commit to git, then:

$ eb deploy

Don’t worry, it doesn’t take quite as long to update (but it can take a while). It does update via deltas (I believe) so it can be pretty quick – you may want to think about having 2+ instances and doing a rolling update if you want to avoid any downtime at all.

Step 7: Add Tuckey URL rewrite for cfwheels:

If you’ve followed the above steps, adding url rewriting is pretty easy.

Download and install the .jar file – instructions here: http://tuckey.org/urlrewrite/

Then add this example set of rules: https://gist.github.com/neokoenig/9d467c585f90292a6a8a

Then git commit, and eb deploy. Note how I put health.html as an exception condition here (see below).

Adding a Production branch:

Is as simple as creating a new branch like before, but doing “production” and then creating a production environment. As your logic to switch datasources is in the app itself, you can just merge staging into production and deploy really quickly. Remember to add any new environments to your RDS security group.

Other notes:

One of the issues I hit was the EB Health monitoring, with requests from Tomcat not containing the correct headers to satisfy the monitoring script. This meant an otherwise working environment would register as “Red”, when actually it was fine. To help with this, I just created an html file in the root called ‘health.html’ which just contains minimal HTML markup, and pointed the health check to /health.html. YMMV.

Remember, sessions from a database will incur more database requests – obvious huh, but when you’ve got 80 concurrent HTTP requests requesting at least one normal DB call, that’s 160 requests to RDS. You may want to consider whether you app actually needs a session scope at all, or the Ben Nadel trick of setting an ultra short session time (until you properly login) so that server memory gets saved, and that Railo then doesn’t do the ‘dump session in database after 10 seconds’ thing.

If you want to upgrade your instance’s from t1.micro -> t2.micro, you can’t currently do this: you have to create a new environment and then switch environment URLS. So ideally, try and select the appropriate machine size to start with.

Autoscaling is great, but… don’t just throw 2500 requests a second at your environment using loader.io or something. Scaling has triggers, such as network output is over ‘x’ for ‘x’ minutes, so it takes a while to warm up. If you simply shove a tonne of concurrent requests at an instance without setting a minimum number of instances of more than one, or prewarming it somehow, you’ll hit memory limits and it’ll fall over, depending on your machine type. Consider different autoscaling triggers (page latency is a fairly good one) other than network output too. And don’t forget to set the scale-in triggers as well as scale-out. You don’t want to trigger 20 machines and for them to never go back to 1-2.

 

v1.1 RoomBooking system released

This is a rather large update, bringing with it some nice new features such as resource booking, some basic digital signage facilities, and an almost complete rewrite under the hood.

Demo | Github

What’s new in 1.1?

Features:

– New static ‘Day’ view (in additional to all the fullcalendar.js views)
– New display board view (for digital signage)
– New Resource management feature: add resources (i.e, laptops) to be bookable alongside an event (#23)
– Support for ‘unique’ resource booking: i.e, will check certain devices for concurrent use
– Support for limiting a resource to a specific location
– Resources can be grouped i.e, Computers,Audio Visual,Furniture
– Clone event functionality
– Calendar “MaxTime” added
– All Day events now set time to 0 – 23:59 to help with alt displays

JS/CSS:
– Upgraded to Bootstrap 3.3.1
– Upgraded to fullcalendar.js 2.x
– Improved datepickers/colourpickers
– All JS/CSS now generated via grunt
– Frontend package managment now from bower
– Added bootstrap form validation
– jQueryUI no longer required
– [removed]: Bootstrap Theme Support
– [removed]: Bootstrap CDN support

Serverside:
– Moved from cfwheels 1.2 -> 1.3.2
– Large underlying codebase rewrite.
– All controllers, models and most helper functions now in cfscript rather than tags: note this may cause issues if you’re on CF9.
– Support for servers with prefix json
– Various options removed/changed to support moment.js
– Improved form validation
– Most JS moved to a single JS file generated via grunt
– Extraneous folders/files removed (i.e /files/)

cfWheels shortcode plugin

This is a port of Barney’s shortcodes implementation for CFML (itself a port of the wordpress shortcodes concept). He wrote all the tricky stuff, I’ve just refactored into cfscript, stripped out all the CFC aspects and made into a cfWheels plugin. Note, I haven’t bothered with the full cfc implementation with execute, as we’re doing this within a cfwheels app.

See https://github.com/neokoenig/cfwheels-shortcodes

Please note nested shortcodes aren’t currently supported.

Usage

// Add a shortcode
addShortcode("code", callback);

// Process content with shortcodes in
processShortcodes(content);

// Quick dump to view existing shortcodes
returnShortcodes();

Example, adding a shortcode for [test]

To add a shortcode, such as [test], first, you should register the code in /events/onapplicationstart.cfm

addShortcode("test", test_shortcode_callback);

This basically registers [test] as a trigger for test_shortcode_callback(). So next, we need to create that function.

In /events/functions.cfm, add your callback:

// Test Shortcode
function test_shortcode_callback(attr) {
    return "foo";
}

Lastly, wrap any content which you want to parse for shortcodes to with sc_process()

    <cfsavecontent variable="mycontent">
      <p>I am [test], fear me.</p>
    </cfsavecontent>
    <cfoutput>
      #processShortcodes(mycontent)#
    </cfoutput> 

Using passed tag attributes

You can use any data passed by tag attributes via the “attr” struct.

[test foo="bar"]

Will allow you to reference #attr.foo# in your shortcode function.

Wrapping content with tags

To enable a tag to ‘wrap’ about content, you need to make sure the callback function has the content argument

// Example shortcode to wrap content in a div
function test_shortcode_callback(attr, content) {
    var result="";
    result="<div class='test'>" & content & "</div>";
    return result;
}

// Usage:
[test]My Stuff[/test]

More advanced example:

This example renders the markup needed for a bootstrap 3.x panel via including a partial

// addShortcode("panel", panel_callback);

/**
 * @hint Renders a bootstrap panel: usage [panel title="My Title"]Content[/panel]
 **/
function panel_callback(attr, content) {
       param name="attr.title" default="";
       var result="";
       savecontent variable="result" {
         includePartial(partial="/common/widgets/panel", title=attr.title, content=content);
       }
       return result;
   }

Panel partial: (/common/widgets/_panel.cfm)

    <cfoutput>
      <div class='panel panel-default clearfix'>
        <div class='panel-heading'>
          <h3 class='panel-title'>#arguments.title#</h3>
        </div>
      <div class='panel-body'>#arguments.content#</div>
    </div>
    </cfoutput>

The nice part about having this within wheels itself is that you can access any of the wheels functions and helpers, including the params struct (i.e, useful for paging numbers etc), whereas using Barney’s cfc in the /miscellaneous/ folder (and then extending controller.cfc) caused various issues.

Simple Ajax requests with cfWheels and jQuery

Someone asked me via email today – I would like to know how to include a Partial dynamically based on a link/button click;

Well, I thought I’d rustle up the simplest way of doing this I could think of.

This example has three parts to it: an Ajax.cfc controller, an index.cfm file and a _time.cfm partial. I’m assuming you’ve got the index.cfm and _time.cfm files in /views/ajax/ and the controller file will live in /controllers/Ajax.cfc

I’ve included two ways of essentially doing the same thing: one renders the partial (which is what the emailer asked for), but depending on what you want to get back, renderWith() can be really useful too, and saves you having to create a partial just to return something simple like a string.

So what’s going on is:

  1. jQuery looks for the class “callremotedata” and takes over its “click” event
  2. It stops the a tag from firing as a normal request
  3. It takes the href value from the a tag
  4. Then fires an ajax request and replaces the HTML in #result with the html returned from the Ajax.cfc controller

Blank HTML emails at outlook.com

Just came across this odd behaviour  – using sendEmail() on a cfWheels app to send registration emails to a hotmail or outlook.com address results in a blank page.

The email and it’s source get through fine, including plaintext variants, but just don’t display.

The culprit? any tags (i.e, <foo>) within a CSS comment. How absurd is that? Well, hopefully it helps someone, somewhere.

Scaling your hosting environment with free services

There comes a point in life when your current hosting environment needs to scale up, and quickly. It might just be that one of your clients gets very popular very quickly, or the gradual adding of sites and resources starts to make services fail. Thankfully there are a few tricks to make life a little bit easier. The less services you can run on your VPS, the more of that precious RAM can get put back into mySQL/Apache etc.

Free Domain based email (https://domains.live.com)
By ‘Domain based email’ I’m referring to sending mail as me@mydomain.com – and not having your webserver or mail client send it ‘on behalf of’, which I think doesn’t look massively professional (and indeed, is a lot more likely to set off the spam filters). Up until a year or two ago, I’d use Google Apps – annoyingly they’ve cut off their free tier; so now the only really free alternative is outlook.com – see https://domains.live.com/signup/signupdomain.aspx – you can do IMAP mail too, so it’s a very useful service. Saves you running your own mail service!

Free Transactional email with Mandrill (http://mandrill.com/
Speaking of sending email via your webserver – ever had mail from your webserver go into people’s junk mail? Ever had your server blacklisted for sending too much mail? Or just been rate limited by Google et al? These are fairly common complaints when running your own VPS: I’ve recently setup Mandrill to deal with all my server-side mail – so far I must say I’m rather impressed. There’s two main ways to do this – either setup your app itself (be it WordPress/Drupal etc) or make your server’s MTA (such as sendmail/postfix) send via the Mandrill SMTP service (or indeed via the Mandrill API directly from your code). Mandrill is free for the first 12,000 mails per month, which will probably be enough for most people I’d wager.

Free Email Troubleshooting (http://mxtoolbox.com/)
This is a stupidly useful tool for checking all things email related, actually, domain related too – try the domain health tool, should hopefully flag up any issues (although some of the warnings are a little strict).

Free AntiSpam service (http://mollom.com/)
It’s not flawless, but if you run a low traffic blog (sub 50 legit posts a day) it’s better than nothing. Has the requisite Drupal/Wordpress modules too.

Free CDN and DNS from CloudFlare (www.cloudflare.com)
This is a bit of a no brainer if you’re not using SSL – change your nameservers over to cloudflare, and you can take advantage of some basic threat management, and a free automatic CDN for static assets (not HTML). Just remember to install the apache/wordpress/drupal plugin to get the referring IP address back from CloudFlare – this is a reverse proxy, so if you don’t understand the implications, do a bit of reading first. You get some (very) basic analytics too, which leads me to..

Free webstats via Google Analytics (www.google.com/analytics)
OK, so Google Analytics has been around for ever, and although Google probably uses your data for everything, it’s still a very useful service indeed. Stats, stats and more stats. Saves you running a serverside stats package (unless of course, you need to track things like direct access PDF downloads – although there are some tricks via URL rewriting which can get around that).

Free Web page testing (http://www.webpagetest.org/)
This is really useful when you’ve setup a site on cloudflare, and you want to test the reverse proxy – but just generally, it’s one of the best site insight tools I’ve seen in terms of loading of assets, working out where they’re coming from etc.

So it’s amazing what you can get for nothing these days: naturally, all these services have their limitations – i.e, 12k emails a day from Mandrill, no SSL for the free version of CloudFlare, etc, but for a lot of websites out there this won’t be an issue.

Mura CMS vulnerability – might apply to your cfWheels app

Just a head’s up:
This recent Mura CMS issue is something which could possibly apply to a wheels app:

http://www.trunkful.com/index.cfm/2014/1/29/Mura-CMS-Vulnerability

I’m not referring to the file manager upload bit (although that could possibly apply), but the role escalation. That is, rewriting POST variables and resubmitting in order to gain admin privs.

Ask yourself, “do I have a public user account creation form” – if not, and user creation is always by an admin level user to start with (and that the user controller is sufficiently locked down), you’re probably fine.

But if you have a user creation form which is publicly accessible,  have you used cfWheels protected properties for the ‘role’ field, or something like ‘isAdmin’ field?

Have a think how but submitting a custom POST request, but simply changing role=”admin” could affect your application. I know I’ve been guilty of this in the past! One quick solution if you don’t want to go down the protected properties route is to separate out your role functions into a separate view/function, which can then do specific checks, and in the user account creation, overwrite any role value passed in with a sensible default.

There’s more on this here – https://groups.google.com/forum/?fromgroups#!searchin/cfwheels/mass$20assignment/cfwheels/bOQo9-CHJlA/xtAVjcvzNMgJ 

This isn’t an issue with wheels per se, just a very common coding practice which can potentially leave a hole open to attack!