Railo and Amazon S3

I don’t usually get the time to play around with Amazon’s cloud offerings, but I’ve started a project recently which has a need for CloudFront, Amazon’s Flash Media Server streaming solution.

I remembered that Railo 3.x and upwards has S3 support (which CloudFront uses as it’s storage) – I wasn’t really prepared for quite *how* easy this was to implement. As a test, all I wanted to do was build a list of files on a S3 bucket, and then construct some URLs for an MP3 playlist in flowplayer.

Firstly, you need to get your Amazon access keys (which I’ll assume you know how to do) via the AWS console.

Next, add this to your onApplicationStart() method (or on cfWheels, your events/onApplicationStart.cfm):

<cfscript>
application.s3.accessKeyId = "MYACCESSKEY";
application.s3.awsSecretKey = "MYSECRETKEY";
application.s3.host="s3.amazonaws.com";
</cfscript>

To get all the files in my S3 bucket, I can then simply do this:

<cfscript>
S3Key=application.s3.accessKeyID;
S3Sec=application.s3.awsSecretKey;
</cfscript>

<cfdirectory action="list"
directory="s3://#S3Key#:#S3Sec#@s3.amazonaws.com/BUCKETNAME/FOLDERNAME" name="s3Directory"
recurse="yes" type="file">

Then, for flowplayer, I needed that data to be looped out into a JS friendly format for the playlist (RTMP streaming uses the mp3: prefix, and I also need to strip out the secret/access key from the returned directory path)

<cfloop query="s3Directory">
<cfoutput>
{url: "mp3:#replace(directory, "s3://#S3Key#:#S3Sec#@/BUCKETNAME/", "", "all")#/#replace(name, '.mp3', '', 'all')#"}<cfif s3Directory.recordcount NEQ currentrow>,</cfif>
</cfoutput>
</cfloop>
</cfsavecontent>

Then in my flowplayer config, I can just add:

playlist: [#playlist#]

… as an option.

Nice and simple, and made v easy by Railo – thanks guys!

Build your own interactive map

The Oxford Internet Institute have been playing around with their own Visualization Library which is out there as open source. One of the things I’ve been doing recently is building the back end to a customiser, which takes your data and preferences and gives you a nicely compiled version.

The back end is written in bespoke ColdFusion, but am in the process of moving it over to cfWheels – having a framework with good documentation really helps when there’s more than one person working on it!

Anyway, it’s now in Beta, so give it a shot – http://api.oii.ox.ac.uk/InteractiveVisBuilder/ – as always, any feedback greatly appreciated!

Fun with isotope.js

Just released this – http://www.oxfordmartin.ox.ac.uk/labs/bigquestions – which uses the fab isotope.js library. A bit of fun / experiment – cfWheels powering the back end for that whole site; was stupidly easy to get wheels to return HTML via ajax to populate the grid.

Go have a play – some interesting videos there too.

Creating an Ajax mySQL Full Text Search in cfWheels

I’m quite a fan of mySQL full text search – don’t get me wrong, it’s not exactly Google, and it has some annoying limitations, but for a quick and easy search function, it’s not half bad. It’s particularly useful for intranets, or just listings in admin section when you might have to delve through a few thousand records otherwise.

This example uses cfWheels, and most importantly, the cfWheels plugin Remote Form Helper.

Step One: Set up an index on your Table in MySQL.

Note: you can only do this on myISAM tables, not innodb.
I usually do this is navicat, where it’s very painless to do indeed. (simply right click a table, select Design table, then select indexes: give the index a name, and select the columns in the table you want to search: make sure the index type is set to fulltext).

For the ‘navicat-less’, try something like this:

CREATE TABLE `articles` 
(
  `id` int(11) NOT NULL default '0',
  `title` varchar(125) default NULL,
  `topic` varchar(25) NOT NULL default '',
  `author` varchar(25) NOT NULL default '',
  `createdAt` datetime NOT NULL default '0000-00-00 00:00:00',
  `body` text NOT NULL,
);

Then add the index:

ALTER TABLE articles ADD FULLTEXT(title,body);

The attributes in the Fulltext() function are the column names we wish to search (I believe there’s a limit of 16 columns).

In your mySQL conf file, add this line under the [mysqld] section:
ft_min_word_len=3

This sets mySQL to search for words of 3 characters or more, rather than the annoying default of 4.

Step 2: cfWheels setup
Download and install/activate/configure the Remote Form Helper plugin.
Make sure you remember to add the line:

addFormat(extension="js", mimeType="text/javascript");

To your config/settings.cfm file as mentioned in the plugin documentation, and also include the wheel.jquery.js file distributed with the plugin. (oh and jQuery itself, obviously).

Step 3: Search Views & Controllers

We need to create at least three files.

Firstly, our controller, /controllers/Search.cfc:

<cfcomponent extends="controller">
<cffunction name="init">
<cfscript>
// This provides bit is essential!
 provides("html, json, js"); 
</cfscript>
</cffunction>

<cffunction name="q" hint="The Main Search Router">
<cfscript>
// If search terms is incoming, and is an ajax request
if(structkeyexists(params, 's') AND len(params.s) GTE 3 AND isAjax())
{   
// searchArticles() function returns our HTML directly to the data attribute 
renderWith(data=searchArticles(params.s), template="articles");
}
else
{ renderNothing() };
</cfscript> 
</cffunction>

<cffunction name="searchArticles" access="private" hint="Our FullText Search">
 <cfargument name="s" required="yes" type="string" hint="The Search Term">
 <cfquery name="q" datasource="#application.wheels.datasourcename#" maxrows="50">
 SELECT id, title, topic, author, body, createdAt FROM articles
 WHERE MATCH (title,body) AGAINST (<cfqueryparam cfsqltype="cf_sql_varchar" value="#arguments.s#">);
 </cfquery>
 <cfloop query="q">
  <cfset q.content[currentrow]=formatResult(string=content, highlightT=arguments.s)>
 </cfloop>
 <cfreturn q />
</cffunction>

<cffunction name="formatResult" access="private" returntype="string" hint="Returns a shortened teaser of the main content, with highlighted search terms">
 <cfargument name="string" type="string" hint="The String to truncate">
 <cfargument name="highlightT" type="string" hint="The term to highlight">
  <cfset var newString="">
  <cfset newString=highlight(text=truncate(stripTags(arguments.string), 400), phrases=arguments.highlightT)>
  <cfreturn newString />
</cffunction>

Next, some view files:

/views/search/index.cfm (where our search form and results output will live)

<cfparam name="params.s" default="">       
<cfoutput> 
<!---Form--->
<!--- NOTE THE remote=true attribute!--->
#startFormTag(controller="search", action="q", id="advancedSearchForm", remote="true")#
#textFieldTag(name="s", placeholder="Search..", label="Search For", value=params.s)#
#submitTag(value="Search")#
#endFormTag()#
<!---Output--->
<div id="results"></div> 
</cfoutput> 

/views/search/articles.js.cfm – This formats our returned dataset for return to the page

<!--- loop over and save the output in a variable--->
<cfsavecontent variable="resultSet">
<cfoutput>
<h2>Articles which match your search:  (#arguments.data.recordcount# results)</h2>
<cfloop query="arguments.data">
<cfoutput>
  <h3>#linkTo(text=title, controller="articles", action="view", key=id)#</h3>
  <h4>#Dateformat(createdAt, 'dd mmm yyyy')#</h4>
  <p>#content#</p>
</cfoutput>
</cfloop>
</cfsavecontent>

<!--- Use the plugin to shove back the results to the page --->
<cfoutput>
#pageReplaceHTML(selector="##results", content=resultSet)#
</cfoutput>

That’s it!
So the form at /views/search/index.cfm posts via ajax to the ‘q’ function in /controllers/Search.cfc.
That q() function returns the html via renderWith(), and the remote form helpers pageReplaceHTML() function posts the results to the results div on the calling page.

The nice part about this approach is that you can use that q() part with a switch/case statement to call any function to return data as you need. You could even go to a different search source, such as a Google Search Appliance, which I’ll cover soon.

Quick example of cfWheels hasManyCheckbox()

I pulled this together from existing code for someone on the cfWheels mailing list: it’s a quick example of three models – two main entities with a join table – and using hasManyCheckbox() to update them in a form.

See https://github.com/neokoenig/cfWheelsHasManyExample

Railo 3 Beginners Guide Review

The Railo 3 Beginner’s Guide is a book released late last year which I’ve just managed to get my hands on thanks to Packt publishing. The core authors include some Railo team members – Mark Drew, Gert Franz and ardent Railo community members Paul Klinkenberg & Jordan Michaels – so there’s no better source here – straight from the horse’s mouth!

Whilst this one is a ‘beginner’s’ guide, it does cover a fairly wide range of topics for developers – they are unarguably ‘Railo-centric’ – but that’s sort of the point.

Chapter 1: Introducing Railo Server
Chapter 2: Installing Railo Server
Chapter 3: CFML Language
Chapter 4: Railo Server Administration
Chapter 5: Developing Applications with Railo Server
Chapter 6: Advanced CFML Functionality
Chapter 7: Multimedia and AJAX
Chapter 8: Resources and Mappings
Chapter 9: Extending Railo Server
Chapter 10: Creating a Video-sharing Application

Generally speaking, I find that books like this have a challenging scope: do you try and fit in everything you know for CFML development? Where do you start/stop? It’s a tricky balance – I found that there were some quite big leaps from absolute basics, like looping & outputting, through to storing information in the ram cache, using ORM and Amazon S3. Don’t get me wrong, these topics are important, but a fair leap away if you’re assuming the user can’t output a loop.

All that said, the examples given are clear and well explained, and certainly give a taste of a Railo outlook on CFML development. Hopefully there will be an advanced series too, which delves down into JVM tuning and garbage collection in details amongst other things.

As far a Railo resource, it’s undoubtedly useful – in fact I would say it’s ideal with a CF developer with ‘some’ experience, who wishes to use Railo-tastic features, and is moving from ACF. However, it’s not something I’d recommend to those who are picking up CF for the first time, as you just can’t fit the entire CFML programming fundamentals into a single book – in my opinion anyway!

Nested Layouts in CFWheels

Must confess I’ve been struggling with this one today, even with the great documentation at cfwheels.org.

Thanks to the cfwheels list, I’ve cracked it though (blogging the solution for posterity!).

I wanted to created a truly nested layout, i.e a master layout with <html> & <body> tags, and then a sub layout in my /views/main/ (with main being the controller name in this instance) which then included the default content on /views/main/index/. The problem was that when you add a layout file in /views/main/layout.cfm it overrides the default layout in /views/layout.cfm.

Turns out you need to call the parent template in the sub template and inject the content using contentFor() before you make the call.

So in my /views/layout.cfm

<html>
<body>
<div id="master-wrapper"> 
<cfoutput>#includeContent("mainbody")#</cfoutput>
</div>
</body>
</html>

and in my /views/main/layout.cfm

<cfoutput>  
<cfsavecontent variable="mainbody">
<div id="controller-layout">
#includeContent()#
</div>
</cfsavecontent>
<!--- Inject the above variable into the parent layout --->
<cfset contentFor(mainbody=mainbody)>
<!--- Include the parent layout --->
#includeLayout("/layout.cfm")#
</cfoutput>

and in my standard view of /views/main/index.cfm

<div id="main-view">
<p>I am the content in the main index file</p>
</div>

Results in…

<!-- Master Layout Start -->
<html>
<body>
<div id="master-wrapper"> 
  <!-- Sub Layout Start -->
  <div id="controller-layout">
    <!-- Standard View Start -->
    <div id="main-view">
    <p>I am the content in the main index file</p>
    </div>
    <!-- Standard View End -->
  </div>
  <!-- Sub Layout End -->
</div>
</body>
</html>
<!-- Master Layout End --> 

Notes: this means you’d need a layout.cfm in every views folder just to set the mainbody variable, otherwise your content never gets called by the top level layout. I’ve tried getting this working with the default body variable but haven’t had much success yet.

Changing a few things!

Sorry to confuse everyone, but I’ve been changing some things… please bear with me whilst I move over posts/comments etc from the old blog. I’ll be adding redirects soon.

I decided to move over WordPress from Mango Blog – that may make the more staunch CF developers in the world shudder, but Mango Blog just wasn’t keeping up with the times for me 🙂

Besides, the right tool for the right job eh?