Quick example of cfWheels hasManyCheckbox()

I pulled this together from existing code for someone on the cfWheels mailing list: it’s a quick example of three models – two main entities with a join table – and using hasManyCheckbox() to update them in a form.

See https://github.com/neokoenig/cfWheelsHasManyExample

Railo 3 Beginners Guide Review

The Railo 3 Beginner’s Guide is a book released late last year which I’ve just managed to get my hands on thanks to Packt publishing. The core authors include some Railo team members – Mark Drew, Gert Franz and ardent Railo community members Paul Klinkenberg & Jordan Michaels – so there’s no better source here – straight from the horse’s mouth!

Whilst this one is a ‘beginner’s’ guide, it does cover a fairly wide range of topics for developers – they are unarguably ‘Railo-centric’ – but that’s sort of the point.

Chapter 1: Introducing Railo Server
Chapter 2: Installing Railo Server
Chapter 3: CFML Language
Chapter 4: Railo Server Administration
Chapter 5: Developing Applications with Railo Server
Chapter 6: Advanced CFML Functionality
Chapter 7: Multimedia and AJAX
Chapter 8: Resources and Mappings
Chapter 9: Extending Railo Server
Chapter 10: Creating a Video-sharing Application

Generally speaking, I find that books like this have a challenging scope: do you try and fit in everything you know for CFML development? Where do you start/stop? It’s a tricky balance – I found that there were some quite big leaps from absolute basics, like looping & outputting, through to storing information in the ram cache, using ORM and Amazon S3. Don’t get me wrong, these topics are important, but a fair leap away if you’re assuming the user can’t output a loop.

All that said, the examples given are clear and well explained, and certainly give a taste of a Railo outlook on CFML development. Hopefully there will be an advanced series too, which delves down into JVM tuning and garbage collection in details amongst other things.

As far a Railo resource, it’s undoubtedly useful – in fact I would say it’s ideal with a CF developer with ‘some’ experience, who wishes to use Railo-tastic features, and is moving from ACF. However, it’s not something I’d recommend to those who are picking up CF for the first time, as you just can’t fit the entire CFML programming fundamentals into a single book – in my opinion anyway!

Nested Layouts in CFWheels

Must confess I’ve been struggling with this one today, even with the great documentation at cfwheels.org.

Thanks to the cfwheels list, I’ve cracked it though (blogging the solution for posterity!).

I wanted to created a truly nested layout, i.e a master layout with <html> & <body> tags, and then a sub layout in my /views/main/ (with main being the controller name in this instance) which then included the default content on /views/main/index/. The problem was that when you add a layout file in /views/main/layout.cfm it overrides the default layout in /views/layout.cfm.

Turns out you need to call the parent template in the sub template and inject the content using contentFor() before you make the call.

So in my /views/layout.cfm

<html>
<body>
<div id="master-wrapper"> 
<cfoutput>#includeContent("mainbody")#</cfoutput>
</div>
</body>
</html>

and in my /views/main/layout.cfm

<cfoutput>  
<cfsavecontent variable="mainbody">
<div id="controller-layout">
#includeContent()#
</div>
</cfsavecontent>
<!--- Inject the above variable into the parent layout --->
<cfset contentFor(mainbody=mainbody)>
<!--- Include the parent layout --->
#includeLayout("/layout.cfm")#
</cfoutput>

and in my standard view of /views/main/index.cfm

<div id="main-view">
<p>I am the content in the main index file</p>
</div>

Results in…

<!-- Master Layout Start -->
<html>
<body>
<div id="master-wrapper"> 
  <!-- Sub Layout Start -->
  <div id="controller-layout">
    <!-- Standard View Start -->
    <div id="main-view">
    <p>I am the content in the main index file</p>
    </div>
    <!-- Standard View End -->
  </div>
  <!-- Sub Layout End -->
</div>
</body>
</html>
<!-- Master Layout End --> 

Notes: this means you’d need a layout.cfm in every views folder just to set the mainbody variable, otherwise your content never gets called by the top level layout. I’ve tried getting this working with the default body variable but haven’t had much success yet.

cfWheels – Active Directory / LDAP authentication

Adding login and administrative features to your cfWheels apps has cropped up on the mailing list a few times, so I thought I’d just pull together a simple example for Active Directory / LDAP authentication.

In this particular snippet, I want to establish a user a) has credentials on the server, and b) belongs to a group called ‘ContactsDatabaseUsers’.

This method has the benefit of a) not having to store user’s passwords in your application database and b) allowing system administrators to control access to the application.

This example assumes you have a login form at /main/login/ which posts params.username and params.password to /main/dologin/

I also would have a local users model where I could add additional application specific options: i.e, when they last logged in, etc.

This information would then be copied to the session scope. ie. session.currentuser

When the users first login, this local entry needs to be created, or copied to the session scope from the existing entry.

<--------Main.cfc----------->
<cffunction name="init">
<cfscript>
// AD Auth, application wide: if I wanted to only restrict a subset a pages, I could use 'only' instead of except below.
// This filter call should be in every controller you wish to protect
filters(through="loginRequired", except="login,dologin");	
// Login
verifies(only="dologin", post=true, params="username,password", error="Both username and password are required", handler="login");	
</cfscript>
</cffunction>

<cffunction name="dologin" hint="Logs in a user via LDAP"> 
  <cfset var ldap=QueryNew("")/>
  <!--- The LDAP Server to authenticate against--->
  <cfset var server="ldap.server.domain.com">
  <!--- The start point in the LDAP tree --->
  <cfset var start="OU=People,DC=domain,DC=com">
  <!--- Scope --->
  <cfset var scope="subtree">
  <!---- Name of group --->
  <cfset var cn = "ContactsDatabaseUsers">
               <cftry>
                   <!--- Check the user's credentials against Windows Active Directory via LDAP--->
                   <cfldap
                       server="#server#" 
                       username="DOMAIN#params.username#" 
                       password="#params.password#" 
                       action="query" 
                       name="ldap" 
                       start="#start#" 
                       scope="#scope#" 
                       attributes="*" 
                       filter="(&(objectclass=*)(sAMAccountName=#params.username#))"/>
                      	
                       <!--- Get the memberOf result and look for contactsDatabase---> 
                       <cfquery dbtype="query" name="q">
                       SELECT * FROM ldap WHERE name = <cfqueryparam cfsqltype="cf_sql_varchar" value="memberOf">
                       </cfquery>
                                             
                       <cfscript>
		if (q.value CONTAINS "CN=#cn#")
			{
			params.name=params.username;
			// Check for the local user profile
			if( model("user").exists(where="username = '#params.username#'"))
			{
				// User found, copy user object to session scope
				user=model("user").findOne(where="username = '#params.username#'"); 
				user.loggedinAt=now();
				user.save();
				session.currentuser=user;                              
				flashInsert(success="Welcome back!");
			}
			else
			{
				// No Local Entry, create user as LDAP Auth has been successful
				user=model("user").create(username=params.username, name=params.name, loggedinAt=now());
				if (NOT user.hasErrors()){
					user.save();         
					session.currentuser=user;							
					flashInsert(success="Welcome - as this is the first time you've logged in, your account has been created.");
				}
				else
				{
					flashInsert(error="Error creating local account from successful Active Directory authentication");
					StructClear(Session);
					renderPage(action="login");
				}
			}
			redirectTo(route="home");
		}
		else 
		{
		 redirectTo(route="home");
		 }						
      </cfscript> 

   <cfcatch type="any"> 
      <cfscript>
           if(FindNoCase("Authentication failed", CFCATCH.message)){
            // Likely to be user error, e.g. typo, wrong username or password
             flashInsert(error="Login Failed: please check your username and password combination");
             renderPage(action="login");
            }
            else {
            // Likely to be LDAP error, e.g. not working, wrong parameters/attributes
            flashInsert(error="Login Failed: Please try again.");
            renderPage(action="login");
            }
       </cfscript> 
   </cfcatch>
 </cftry>                       
</cffunction>
</cfcomponent>

<-----------Controller.cfc------------->
<cffunction name="loginRequired" hint="This function will prevent non-logged in users from accessing specific actions">
<cfif NOT isloggedin()> 
	<cfset redirectTo(controller="main", action="login")>
</cfif>
</cffunction> 
    
<cffunction name="isloggedin" hint="Checks for existence of session struct">
<cfif NOT StructKeyExists(session, "currentUser")> 
   <cfreturn false>
<cfelse>
   <cfreturn true>
</cfif>
</cffunction>

Don’t forget nested functions in CFWheels

Occasionally I forget clever little things, like using functions as argument values.

Here’s a small snippet which I ended up using when doing a blog front page, where I wanted to strip the tags out of the blog body, truncate it to 300 characters, and then add a ‘read more’ link at the end of the paragraph with an ellipsis.

There’s a load of nice little functions in CFWheels for this – stripTags(), truncate() and linkTo().

Here’s what the helper in the loop ended up as:

#truncate(
  text=stripTags(body), 
  length=300, 
  truncateString="#linkTo(
      text='... Read More &raquo;', 
      route='blog', 
      action='view', 
      key=id)#"
)#

Password Hashing and Salting

As I’m (hopefully) going to release my first open source project soon, I thought it would be a good time to revisit application security, specifically password hashing, salting and general encryption. If you’ve not come across Jason Dean’s blog, I’d recommend that as a first port of call, as most of what I’m referring to is explained there.

Password security is one of those never ending journeys; you have to assume that it’ll never be perfect, and that there’s always room for improvement. I’d recently had an idea to improve password hashing a little, which someone might find useful in principle.

Let’s take a simple password, like ‘myPassword1’ (incidentally, if you actually have any passwords like that, please go and get lastpass or password1 immediately and wash your keyboard out with soap).

‘myPassword1’ stored in a database in plain text is obviously a very, very bad idea. Most people’s gut reaction would be to hash it, which at least is one way.

<cfset thePassword='myPassword1'>
<cfset passwordHash=Hash(thePassword, 'SHA-512') />

But wait!, let’s assume 90% of our users are stupid, or at least, uninformed. If two people have the same password, their hashes will appear identically in the password column. On top of that, if I hash the password again, I’ll get exactly the same resultant string. So a hashed (unsalted) password is really not that useful. You could perform a ‘rainbow table’ attack: password hashes from your database can be compared against a table of known hashes (such the hash for myPassword1), et voila, password revealed.

So the next step is to introduce salting: that is, appending or prepending a unique string to the password to make the resultant hash unique, and make rainbow table attacks much much harder.

<cfset salt=createUUID() />
<cfset passwordHash=Hash(thePassword & salt, 'SHA-512') />

This has the advantage that attacks require the salt to even attempt some sort of dictionary/rainbow table attack. Usually the salt is stored alongside the hashed and salted password in the database.

This is the part which I think could be improved by a relatively small step. By storing the salt alongside the password in the users table, we’re essentially giving a hacker the necessary ammunition to attempt to compromise the password. Let’s make it a little harder. Let’s encrypt the salt itself using another key, stored outside the webroot. So I might have a folder called ‘auth’ on the same level as my ‘webroot’ or ‘html’ folder. In there, I’m going to store a text file with a UUID inside. When I want to compare a password hash, I now have to decrypt the salt before I can use it (as I acutally need the salt value, I’m not hashing it, as that’s one way) using the key read in via CFFILE outside the webroot.

<!---Get the key--->
<cfset authKeyLocation=expandpath('../auth/key.txt')>
<cffile action="read" file="#authKeyLocation#" variable="authkey">
<!--- New password hashing --->
<!--- Generate a salt, this is never stored in it's plain form---> 
<cfset theSalt=createUUID() />
<!--- Hash the password with the salt in it's plain form---> 
<cfset passwordHash=Hash(thePassword & theSalt, 'SHA-512') /> 
<!--- The encrypted salt to store in the database, 
using the authKey--->
<cfset salt=encrypt(theSalt, authKey, 'CFMX_COMPAT')>

So why do this? If your database is compromised, you at least have an additional key to the puzzle missing for the attacker: they would have to do both a compromise of the filesystem AND the database to get anywhere. Additionally, you could quickly invalidate all the passwords in your application by deleting/regenerating the ‘master key’ at a file system level (this also assumes you have a half decent ‘forgotten/reset password’ feature in your application), which could be useful is you suspect the database and file system to be compromised at any point.

Whilst this is hardly comprehensive explanation, by adding additional layers and parts to the process, we’ll hopefully make any attackers lives a little more difficult. Jason Dean adds another step I like looping the hashing process over 1000 times – if you made this number completely arbitary between 1000 and 1500, you’d have another variable the attacker would have to get, and this one would be stored in the source code of the application itself.

So assuming you added all the above measures, you’d have to:

  1. compromise the database (for the password hash, and the encrypted salt),
  2. the source code for the application (to get the number of hashing loops), and
  3. get the facility to navigate outside the (potentially) locked webroot (to get the master key to decrypt the encrypted salt).

Mucking around with numbers

So, I was reading up on the Fibonacci sequence the other day, and wondered how long it would take CF to calculate the sequence to a couple of thousand steps, and also how long those numbers ended up becoming.

Consider the following loop:

<cfset n1 =0>
<cfset n2 =1>
<table>
<thead>
<tr><th>Row</th><th>No.</th><th>Length</th>
</thead>
<tbody>
<cfloop from="1" to="1000" index="i">
<cfset x = (n1 + n2)>
<cfoutput>
<tr><td>#i#</td><td>#len(numberformat(x))#</td><td>#numberformat(x)#</td></tr>
</cfoutput>
<cfset n1 = n2>
<cfset n2 = x>
</cfloop>
</tbody>
</table>

I immediately hit a few issues. One was scientific notation, which was solved by numberformat(), but the other issue I found was that if I tried to increase the number of loop iterations beyond 1475, I’d get an error simply stating ‘For input string: “I”‘

Removing the numberformat mask revealed 1.30698922376E+308 as the largest generated number in the sequence, with the remaining rows represented as ‘1.#INF’.

So, out of curiosity, does anyone know why? Is it simply that Java can’t handle larger integers?

Linode Vs VPS.net: the signup.

I’ve been on a quest for a new VPS, specifically, I wanted something for a LAMP stack – on top of all my CF sites, I’ve got Drupal/Wordpress/PHPBB3 to host as well.

After a bit of research, I narrowed my choices down to Linode and VPS.net. I also wanted a XEN based VM, as I’ve already got a Virtuozzo VM; there’s one thing which scares me a bit about a virtuozzo based vm, and that’s the lack of swap. Whilst the counter claim is that swap is slow compared to actual RAM, virtuozzo based containers give you no leeway there – they just seem to run into out of memory errors.

Linode:

Linode offer a 7 day money back offer, so I thought I’d give them a try.

The signup process was easy, but when I tried to actually create my disk images and build my distro, I just got ‘Create Disk Failed’. Pleasingly, an IRC client hooked into the Linode channel is built into the admin interface.

I went on an said ‘eh? just signed up but..’ Another customer told me to raise a ticket, which I did. I wasn’t really expecting much, I’d sort of resigned myself to the fact that I was in for some pain and a least 4 hours wait.

4 minutes later, it was fixed, with an apology – they’d simply migrated my linode to a new host, and off I went. 4 minutes for a fix is insanely fast (amusingly, my incredulation was met with ‘Oh year, Peter is a ticket ninja – 4 minutes is kinda slow for him..’ on IRC.)

The interface works well. I’ve already trashed 5 Ubuntu servers, and a couple of Centos ones for good measure, just to try out the preconfigured ‘stack scripts’ they’ve got – i.e Hardened PHP & LAMP stack etc..There’s user generated ones too.

So I was up and running pretty much instantly.

I think my only complaint so far (and it’s not *really* a complaint) is I couldn’t quite get the Stackscript with LAMP stack and firewalling going properly (although this is probably more a reflection on my linux skills than the script), so it meant some manual IPTABLES config. Firewalling is one of those things I prefer to do via GUI (I know, I know, get some CommandlineFu already) in VZ, your firewall is essentially forwarded, so you can configure it outside the VPS, which is a nice trick.

Running a small test via loadimpact had a heavily loaded Drupal site coming in at 800ms for 50 concurrent users, which isn’t bad. By comparison, the current shared host that site is on, the tests wouldn’t go above 30 concurrent users, as it had gone from 1.5 seconds (10 users) to greater than 14 seconds (20 concurrent). No match really. But then, that’s expected. RAM idled around 245MB during the test, with a CPU over 4 cores not really exceeding about 20%. All that’s with out of the box apache/mySQL, no real tuning at all.

vps.net

I’d had a fairly good response to twitter queries from @vpsnet, and thought they might be worth a try, as I really like the concept of scalable expandable hosting (i.e adding a node).

The signup process was fine at the start, once I got to a computer where the VPS.net site didn’t think I was in the States. Sorry guys, but the University of Oxford is *not* in the US: either way, not having a way of switching currency or setting your region is bad. So at home, I managed to actually get billed in GBP and signed up.

Instantly, my account was flagged as being ‘held for validation’. 13 hours later, after a ranting tweet, it was magically approved. Still, that’s potentially a lot of time wasted right there. I’d even submitted a ticket, where it was still open 14 hours later, with ‘Private Staff Comment’ as the last response. That’s sort of annoying too.

Then, trying to pay the initial invoice, neither my CC nor PayPal work. Their signup/payment/check process apparently does a micro transaction first, followed by the larger amount; this triggered my bank’s anti fraud measures. I tried two cards, (and also paypal, but of course, this was linked to my card..) and managed to trigger the anti fraud measures on both of them.

After all that, I sort of lost the will, so frankly, deleted my account (not paid anything, or even tried anything after all by this point).

Maybe one day I’ll try again. I will note, that @vpsnet were helpful on Twitter, and I even got a phone message from the states checking on my progress (!), although by that point I’d sort of given up, so I think there is some genuine promise in their customer support.

Oh well vps.net, maybe another day.

I ended up keeping the Linode VPS, and am now addicted to looking at CPU/RAM/IO graphs. Sad huh. Oh, and htop – that’s addictive too.

And lastly, a shameless referal link for linode. Go on, you know you want to.

CFWheels in Use at the OII

Obviously, I’ve been a fan of CFWheels for a fair while now. So naturally, when the opportunity came to build a new app from scratch, there was only one real contender.

At the Oxford Internet Institute (OII) – where you can find me from 9-5 (most of the time) – we’ve primarily been a research department, with no students. This changed a few years ago, with the introduction of a DPhil programme, and more recently, an MSc programme.

Naturally, when you throw 30 students a year together, and another 20 or so tutors/lecturers, you’re going to need some sort of organisational/courseware app. For the first year or two, we tried Wikis, Sakai, trialled Moodle, kept well away from Blackboard (due to licensing) but none of them quite did what we wanted them to. They all were built for another purpose – delivering complete courses online, or managing things we didn’t need, creating a lot of UI clutter.

Enter the new CFWheels app, nicknamed ‘Plato’ (not officially called Plato, as I think there’s *already* some existing courseware with that name). Let’s call it cfPlato then.

The aim was to build a closed system which fullfilled the following:

  • To list courses, their outlines and who’s teaching them
  • For Tutors to communicate with their students via bulk email
  • A complete email archive for all courses and groups
  • File storage, with archiving, tagging and bulk downloading via .zip
  • Individual user accounts
  • File submission
  • Deadline listings with notifications
  • Threaded discussion via Forums
  • Facilities for smaller groups, similar to courses
  • Multiple user roles, such as tutor, student, administrator etc.
  • Full logging and audit trail
  • Avatar uploading and cropping
  • Ability for students to provide anonymous Course feedback

I can up with the idea that we should give access to pretty much everything to authenticated users, then allow student to ‘subscribe’ to their areas of interest; Subscribing means they get email notifications etc, and various shortcuts in the system to their courses of choice. Whilst this puts the responsibility of subscribing to courses in the hands of the students, it removes a large administrative overhead.

As this is currently a closed system, I can’t demo it properly, but I thought it might interest other people to get an idea of the sort of apps which are starting to spring up based on CFWheels.

Tonnes of screenshots here as a Flickr slideshow. Turn on Descriptions/comments for some more info:

http://www.flickr.com/photos/oxfordalto/sets/72157625977093921/show/

Built using CFWheels, CF8, BlueprintCSS, JQuery, JQuery Tools, TinyMCE, JCrop.

Comments & Questions welcome!

Building a Webcasting Troubleshooter

At the Oxford Internet Institute (OII), we run a fair few services, one of which is a streaming service based on Wowza Streaming Server. Wowza is a great piece of software (once configured properly) and allows us to stream .h264 encoded mp4s, and also stream live.

Recently, we’ve been trialling the live service as a bit of a recruitment tool; as a department which studies the social effects of the Internet, it seems logical we should utilise social media and streaming video as a way of attracting students, but also to help with the open door effort of not excluding anyone who may wish to apply.

One of the difficulties in providing live webcasting services is the lack of control over the end user’s experience. By using an .h264 encoded stream over RTMP, there are certain conditions which need to be met on the client end. Testing for these conditions has recently proved a bit of a head-scratcher, but we’ve come up with a small testing suite which at least helps push the user in the correct direction.

So what do we need to test?
    • Firstly, Javascript has to be enabled;
    • Secondly, we need to test for Flash, and preferably a version number,
    • And lastly, we need to test that port 1935 is open on the client end, for streaming over RTMP.

The end user should hopefully see three check boxes, for Javascript, Flash and finally, the port 1935 tester. If any of the tests fail, then they should be directed to instructions for enabling Javascript, updating Flash, etc.

I won’t go into the details of the Javascript/Flash detection here, as it’s generally well documented on the web.

The challenge was the port 1935 check.

We didn’t want to go down the Java Applet route (as that would lead to yet another condition to test for), and we didn’t want to give end users instructions on how to ping / tracert etc, as that would simply be too complex for the majority of users. We wanted a solution which would integrate with the rest of the page easily.

As we just need to check 1935 is open from the client end, *any* response on 1935 can be measured as a successful connection. Wowza doesn’t run a HTTP server in the truest sense – it *does* however respond with the Wowza streaming server number on port 80, and also on 1935; It’s a seemingly non standard response though, at least as far as the http headers are concerned.

Via ColdFusion?

So initially, I wasn’t thinking straight, and suggested doing a CFHTTP request to the streaming server on port 1935 to try and gauge a response. As the returned request is slightly odd, CFHTTP threw a few parsing errors, and I ended up trying to get the page as a binary object, which worked. After all, at the time, I was just looking to get anything from the streaming server over 1935.

Obviously, that request would come from our webserver, not the client, so pretty useless for testing whether the  end user could connect on 1935. So that rules out Server side scripting straight away.

Via jQuery?

Next, jQuery was the obvious candidate: it’s a client-side language after all. Couldn’t we just do a get request via jQuery, and gauge the response that way? Unfortunately (well, fortunately for the world wide web) there’s a fair amount of browser restrictions on cross domain requests when it comes to Javascript. You can’t just grab a page hosted on another domain, on a non standard port that way, and access it via jQuery.

Via iFrame?

We can’t use an iFrame, as the browser won’t allow you to access the content from another domain, so passing the result to jQuery wasn’t possible. Also, returning the Wowza generated page wouldn’t be exactly user friendly, and a line such as ‘If you don’t see Wowza X.XXX.XXX below, there’s a connection issue’ isn’t a particularly useful response.

Via JQuery & JSONP?

JSONP (JSON with Padding) is the answer in this case – you could get jQuery to request JSONP – However, we really didn’t want to have to configure Wowza with it’s fairly complex configuration to return JSONP (I’m not even sure if it’s possible, as the Wowza generated page comes from a load of XML config, but it may well be). We also didn’t want to do anything to the rudimentary http server which might affect the video streaming.

Via Proxy?

We couldn’t also do a proxy, as the request would again not be originating from the client – or rather the client request would go over port 80 to the proxy, and then the proxy would continue on 1935. Same issue as Server side scripting.

Ok, Flash Proxy?

The one advantage is that we had control over the server we wanted to test. Enter flash proxy; there’s a good guide here: http://usejquery.com/posts/9/the-jquery-cross-domain-ajax-guide .

By opening up the crossdomain.xml on the streaming server (but restricting it to requests from our domain), we can allow flash to do the check and return the result to jQuery – see flHXR – http://flxhr.flensed.com/ (with the jQuery plugin).

All in all, quite a learning journey!

More links: