Latest version of Flash on Mac breaks RTMP streams

We use RTMP streaming from Wowza server here at the Internet Institute and The Oxford Martin School. Currently, the latest version of flash (the one with hardware accelaration) on Mac seems to be causing headaches for some users, specifically, on 10.6.3 upwards, .h264 encoded streams over RTMP appear to fail.

You’ll get JWPlayer/FlowPlayer loading properly, followed by a blank screen.

Chrome has its own version of Flash, which appears unaffected, but for Firefox/Safari, you may wish to try the following: (I appreciate this is a quick fix, and not a solution!)

  • Right click (control click) on the main player window, and select ‘settings’
  • Ensure the ‘display’ tab is selected (bottom left of the dialog window)
  • Untick ‘Enable Hardware Acceleration’ and click close.
  • Reload the page

If anyone has an actual fix for this, do let me know…!

Linode Vs VPS.net: the signup.

I’ve been on a quest for a new VPS, specifically, I wanted something for a LAMP stack – on top of all my CF sites, I’ve got Drupal/Wordpress/PHPBB3 to host as well.

After a bit of research, I narrowed my choices down to Linode and VPS.net. I also wanted a XEN based VM, as I’ve already got a Virtuozzo VM; there’s one thing which scares me a bit about a virtuozzo based vm, and that’s the lack of swap. Whilst the counter claim is that swap is slow compared to actual RAM, virtuozzo based containers give you no leeway there – they just seem to run into out of memory errors.

Linode:

Linode offer a 7 day money back offer, so I thought I’d give them a try.

The signup process was easy, but when I tried to actually create my disk images and build my distro, I just got ‘Create Disk Failed’. Pleasingly, an IRC client hooked into the Linode channel is built into the admin interface.

I went on an said ‘eh? just signed up but..’ Another customer told me to raise a ticket, which I did. I wasn’t really expecting much, I’d sort of resigned myself to the fact that I was in for some pain and a least 4 hours wait.

4 minutes later, it was fixed, with an apology – they’d simply migrated my linode to a new host, and off I went. 4 minutes for a fix is insanely fast (amusingly, my incredulation was met with ‘Oh year, Peter is a ticket ninja – 4 minutes is kinda slow for him..’ on IRC.)

The interface works well. I’ve already trashed 5 Ubuntu servers, and a couple of Centos ones for good measure, just to try out the preconfigured ‘stack scripts’ they’ve got – i.e Hardened PHP & LAMP stack etc..There’s user generated ones too.

So I was up and running pretty much instantly.

I think my only complaint so far (and it’s not *really* a complaint) is I couldn’t quite get the Stackscript with LAMP stack and firewalling going properly (although this is probably more a reflection on my linux skills than the script), so it meant some manual IPTABLES config. Firewalling is one of those things I prefer to do via GUI (I know, I know, get some CommandlineFu already) in VZ, your firewall is essentially forwarded, so you can configure it outside the VPS, which is a nice trick.

Running a small test via loadimpact had a heavily loaded Drupal site coming in at 800ms for 50 concurrent users, which isn’t bad. By comparison, the current shared host that site is on, the tests wouldn’t go above 30 concurrent users, as it had gone from 1.5 seconds (10 users) to greater than 14 seconds (20 concurrent). No match really. But then, that’s expected. RAM idled around 245MB during the test, with a CPU over 4 cores not really exceeding about 20%. All that’s with out of the box apache/mySQL, no real tuning at all.

vps.net

I’d had a fairly good response to twitter queries from @vpsnet, and thought they might be worth a try, as I really like the concept of scalable expandable hosting (i.e adding a node).

The signup process was fine at the start, once I got to a computer where the VPS.net site didn’t think I was in the States. Sorry guys, but the University of Oxford is *not* in the US: either way, not having a way of switching currency or setting your region is bad. So at home, I managed to actually get billed in GBP and signed up.

Instantly, my account was flagged as being ‘held for validation’. 13 hours later, after a ranting tweet, it was magically approved. Still, that’s potentially a lot of time wasted right there. I’d even submitted a ticket, where it was still open 14 hours later, with ‘Private Staff Comment’ as the last response. That’s sort of annoying too.

Then, trying to pay the initial invoice, neither my CC nor PayPal work. Their signup/payment/check process apparently does a micro transaction first, followed by the larger amount; this triggered my bank’s anti fraud measures. I tried two cards, (and also paypal, but of course, this was linked to my card..) and managed to trigger the anti fraud measures on both of them.

After all that, I sort of lost the will, so frankly, deleted my account (not paid anything, or even tried anything after all by this point).

Maybe one day I’ll try again. I will note, that @vpsnet were helpful on Twitter, and I even got a phone message from the states checking on my progress (!), although by that point I’d sort of given up, so I think there is some genuine promise in their customer support.

Oh well vps.net, maybe another day.

I ended up keeping the Linode VPS, and am now addicted to looking at CPU/RAM/IO graphs. Sad huh. Oh, and htop – that’s addictive too.

And lastly, a shameless referal link for linode. Go on, you know you want to.

CFWheels in Use at the OII

Obviously, I’ve been a fan of CFWheels for a fair while now. So naturally, when the opportunity came to build a new app from scratch, there was only one real contender.

At the Oxford Internet Institute (OII) – where you can find me from 9-5 (most of the time) – we’ve primarily been a research department, with no students. This changed a few years ago, with the introduction of a DPhil programme, and more recently, an MSc programme.

Naturally, when you throw 30 students a year together, and another 20 or so tutors/lecturers, you’re going to need some sort of organisational/courseware app. For the first year or two, we tried Wikis, Sakai, trialled Moodle, kept well away from Blackboard (due to licensing) but none of them quite did what we wanted them to. They all were built for another purpose – delivering complete courses online, or managing things we didn’t need, creating a lot of UI clutter.

Enter the new CFWheels app, nicknamed ‘Plato’ (not officially called Plato, as I think there’s *already* some existing courseware with that name). Let’s call it cfPlato then.

The aim was to build a closed system which fullfilled the following:

  • To list courses, their outlines and who’s teaching them
  • For Tutors to communicate with their students via bulk email
  • A complete email archive for all courses and groups
  • File storage, with archiving, tagging and bulk downloading via .zip
  • Individual user accounts
  • File submission
  • Deadline listings with notifications
  • Threaded discussion via Forums
  • Facilities for smaller groups, similar to courses
  • Multiple user roles, such as tutor, student, administrator etc.
  • Full logging and audit trail
  • Avatar uploading and cropping
  • Ability for students to provide anonymous Course feedback

I can up with the idea that we should give access to pretty much everything to authenticated users, then allow student to ‘subscribe’ to their areas of interest; Subscribing means they get email notifications etc, and various shortcuts in the system to their courses of choice. Whilst this puts the responsibility of subscribing to courses in the hands of the students, it removes a large administrative overhead.

As this is currently a closed system, I can’t demo it properly, but I thought it might interest other people to get an idea of the sort of apps which are starting to spring up based on CFWheels.

Tonnes of screenshots here as a Flickr slideshow. Turn on Descriptions/comments for some more info:

http://www.flickr.com/photos/oxfordalto/sets/72157625977093921/show/

Built using CFWheels, CF8, BlueprintCSS, JQuery, JQuery Tools, TinyMCE, JCrop.

Comments & Questions welcome!

Building a Webcasting Troubleshooter

At the Oxford Internet Institute (OII), we run a fair few services, one of which is a streaming service based on Wowza Streaming Server. Wowza is a great piece of software (once configured properly) and allows us to stream .h264 encoded mp4s, and also stream live.

Recently, we’ve been trialling the live service as a bit of a recruitment tool; as a department which studies the social effects of the Internet, it seems logical we should utilise social media and streaming video as a way of attracting students, but also to help with the open door effort of not excluding anyone who may wish to apply.

One of the difficulties in providing live webcasting services is the lack of control over the end user’s experience. By using an .h264 encoded stream over RTMP, there are certain conditions which need to be met on the client end. Testing for these conditions has recently proved a bit of a head-scratcher, but we’ve come up with a small testing suite which at least helps push the user in the correct direction.

So what do we need to test?
    • Firstly, Javascript has to be enabled;
    • Secondly, we need to test for Flash, and preferably a version number,
    • And lastly, we need to test that port 1935 is open on the client end, for streaming over RTMP.

The end user should hopefully see three check boxes, for Javascript, Flash and finally, the port 1935 tester. If any of the tests fail, then they should be directed to instructions for enabling Javascript, updating Flash, etc.

I won’t go into the details of the Javascript/Flash detection here, as it’s generally well documented on the web.

The challenge was the port 1935 check.

We didn’t want to go down the Java Applet route (as that would lead to yet another condition to test for), and we didn’t want to give end users instructions on how to ping / tracert etc, as that would simply be too complex for the majority of users. We wanted a solution which would integrate with the rest of the page easily.

As we just need to check 1935 is open from the client end, *any* response on 1935 can be measured as a successful connection. Wowza doesn’t run a HTTP server in the truest sense – it *does* however respond with the Wowza streaming server number on port 80, and also on 1935; It’s a seemingly non standard response though, at least as far as the http headers are concerned.

Via ColdFusion?

So initially, I wasn’t thinking straight, and suggested doing a CFHTTP request to the streaming server on port 1935 to try and gauge a response. As the returned request is slightly odd, CFHTTP threw a few parsing errors, and I ended up trying to get the page as a binary object, which worked. After all, at the time, I was just looking to get anything from the streaming server over 1935.

Obviously, that request would come from our webserver, not the client, so pretty useless for testing whether the  end user could connect on 1935. So that rules out Server side scripting straight away.

Via jQuery?

Next, jQuery was the obvious candidate: it’s a client-side language after all. Couldn’t we just do a get request via jQuery, and gauge the response that way? Unfortunately (well, fortunately for the world wide web) there’s a fair amount of browser restrictions on cross domain requests when it comes to Javascript. You can’t just grab a page hosted on another domain, on a non standard port that way, and access it via jQuery.

Via iFrame?

We can’t use an iFrame, as the browser won’t allow you to access the content from another domain, so passing the result to jQuery wasn’t possible. Also, returning the Wowza generated page wouldn’t be exactly user friendly, and a line such as ‘If you don’t see Wowza X.XXX.XXX below, there’s a connection issue’ isn’t a particularly useful response.

Via JQuery & JSONP?

JSONP (JSON with Padding) is the answer in this case – you could get jQuery to request JSONP – However, we really didn’t want to have to configure Wowza with it’s fairly complex configuration to return JSONP (I’m not even sure if it’s possible, as the Wowza generated page comes from a load of XML config, but it may well be). We also didn’t want to do anything to the rudimentary http server which might affect the video streaming.

Via Proxy?

We couldn’t also do a proxy, as the request would again not be originating from the client – or rather the client request would go over port 80 to the proxy, and then the proxy would continue on 1935. Same issue as Server side scripting.

Ok, Flash Proxy?

The one advantage is that we had control over the server we wanted to test. Enter flash proxy; there’s a good guide here: http://usejquery.com/posts/9/the-jquery-cross-domain-ajax-guide .

By opening up the crossdomain.xml on the streaming server (but restricting it to requests from our domain), we can allow flash to do the check and return the result to jQuery – see flHXR – http://flxhr.flensed.com/ (with the jQuery plugin).

All in all, quite a learning journey!

More links:

SOTR2010

I’ve just come back from ‘Scotch on the Rocks’ 2010, held in London. This has been quite a step up from last year!

I’ve had the pleasure of hearing Ben Nadel (@bennadel), Ray Camden (@cfjedimaster), Terry Ryan, Gert Franz and Aral Balkan (@aral). All great presentations, with Aral’s sheer passion for design (or should I say, ‘experience’)  coming through in particular.

Ben talked on CF’s Application framework, talking about session manipulation is a way I’d really just never thought of before – by opening the subject up, he definitely made it more accessible, and has some great code examples I must get my hands on.

Ray was on CFBuilder duty, and whilst it is something I *really* should get around to trying, the amount of times he said ‘oh and this doesn’t work, but don’t worry I’ve filed a bug’ did leave me slightly worried 🙂 I enjoyed his Ajax Q&A session too – as something which doesn’t always come into my projects, it was interesting to hear that my thoughts on Ajax generally are pretty well reflected from those that spoke. 

Railo 4 is looking to be stellar; there’s a billion things they’re planning which are a bit beyond me (particularly the whole Java integration and command line stuff) but it’s very encouraging to see Railo doing so well; I’ve been using Railo alongside CF8 for a year now, and it really is a fantastic open source project.

Apart from the 6am ‘get up grab train and run’ (and having to leave early) it’s been a great couple of days. Nice to meet so many of the other CF Twitter(er)s too; not to mention seeing people whose blogs I’ve been reading for 5 years for the first time! Congrats to the whole SOTR team, and all the speakers.

Discovering CFWheels

I’ve spent the past week or so with my head down, coding, and completely loving it.

CFWheels is a CFML framework based on Ruby on Rails – at least it shares a lot of the same concepts from what I can tell. My previous experience with frameworks hasn’t been too great. I got put off the ‘big player’ names like Model Glue, Fusebox et al, as when I investigated them (admittedly a few of years ago) I got instantly put off by XML configuration, and quite frankly, some concepts I just couldn’t grasp at the time. Also, loading 100’s of templates for the apps I was writing at the time was just unnecessary.

Maybe the timing’s good, but I’ve been blown away by CFWheels.

My own coding style has changed a lot in the last 12 – 36 months, and I was getting to the point where I was writing lots of custom code to do certain things – a good example would be URL rewriting – each application I’d write would have similar, if not the same requirements. I’m a big fan of logical and symantically meaningful URL paths, and where I can, I’ve been trying to gradually make all my apps follow some sort of pattern. Once you find something which works, you basically cut and paste it into every new app you make, and each application might it’s own quirks, meaning you have multiple, similar versions of the same block of code.

Basic CRUD operations too – it’s incredible how much code I’ve written over the years which is based around what I’d call a ‘CRUD switch’ – i.e  a cfswitch block with cases of ‘view,add,edit,delete’. Almost without realising it , I’d been gradually moving towards the MVC pattern of coding, it just made sense.

Enter CFWheels: I downloaded it, and started playing. Within about 2-3 hours, I’d mastered enough of the basics to port an existing (simple) site over. I even got to the point where I was using an internal Wheels function, and just *guessed* what the arguments should have been; low and behold, it worked. When you find a framework which actually ‘thinks’ like you do, it’s a pretty great feeling.

My particular highlights thus far:

  • Basic jawdropping when I realised the power of the internal ORM engine.
  • ‘Routes’ – very, very useful.
  • The separate configuration setting options for development/design/production etc
  • Caching is, well, just painless.
  • HasMany() – running out of superlatives, but to be able to say, ‘Departments hasMany People’ and then in the People model, ‘People BelongsTo Departments’, and then refer directly to the relationship without a line of SQL? cool.
  • The Form Helpers save a lot of code.
  • The CFWheels Google group guys have been really helpful, and the documentation is excellent.
  • The Global Helpers bit is very useful, previously, you’d have to cache all your Utility functions in a CFC and chuck it in the Application scope – now you just add it to one place to make it available to all your views.
  • All the internal Wheels functions – when I’d hit a point where I thought ‘there should be a function for this’, there was.
  • It’s fast. Especially on Railo (disclaimer: it just ‘feels’ fast! And no, i’ve not tried it on CF9, and no, I’ve got no figures to back this one up)

Naturally, some low points too:

  • Note to self: don’t casually, or accidently dump extremely large objects on a production server. It appears to eat up RAM very quickly.
  • Associations – amazing concept, but out of everything, took me the longest time to get my head around; then I found ‘shortcuts’ and it all made sense.
  • I did get initially confused when something which I expected to be a query turned out to be an object..(then I found you can override that behaviour!)
  • The routes system could really do with a better ‘catchall’ system – it works, but I don’t want to have to specify all my controllers routes: I want Wheels to introspect them and work it out 🙂 (I’m aware there is talk going on re: this issue)
  • I still want to see some more example applications of best practice / example apps generally.
  • It took me a while to be comfortable with *NOT* scoping everything 🙂

All in all, I’m really enjoying this one, and it’s made me reconsider my previously (slight naïve) position on frameworks.

Well worth a couple of hours of your life to try.

http://cfwheels.org/

You actually have to be impressed by Spammers sometimes…

I’ve just been cleaning up the aftermath of an exploited WordPress install. I officially hate wordpress – this particular install had been updated a few mere weeks before, but an exploit had clearly got out for that version, and therefore – havoc.

On going through the files, database, changing all the passwords etc etc, I got to the point where I was actually quite impressed by what this automated attack had done.

Firstly, exploited files were stored in both /wp-content and elsewhere – traditionally, when you ugrade WordPress, you tend to overwrite everything but the wp-content directory, .htaccess file, and wp-config.php file; the wp-content directory stores uploads, so an upgrade to WordPress wouldn’t automatically overwrite the compromised files.

Secondly, everything was Base64 encoded – once you’ve got the hang of what they’re doing, it’s fairly simple to search for “gzinflate(base64_decode” or some such string, but a typical search for ‘Viagra’ or some such spam word will never show in a search of the source code.

Thirdly – and this one is fairly obvious – every output was hidden by inline CSS, either via “display:none;” or “height:0 width:0” etc. so as not to alert the blog owner.

The really clever bit which got me was the fact that browsing via a normal browser wouldn’t make the spam show up. You had to spoof a Googlebot user agent to see the damage (oh, and turn off CSS as well).

When was the last time *you* checked your site for spam? Google this to find out – ‘site:yourdomain viagra’

OxAlto Capita – A Free Theme for Mango Blog

As I’ve been exploring some more CF open source apps, I’ve come across Mango Blog which I’m really beginning to like.

So, as an attempt to get to the know the bits under the hood, I’ve written a theme which I’ve decided to release under Creative Commons – so that’s free to use for all you Mango Blog users.

It’s based on the Blueprint CSS framework, so hopefully you should be able to take advantage on the various layout schemes incorporated into that.

The banner image remains property of The Hinksey Studio, but has been released for use in this theme.

Edit: This is now available directly via the Mango Blog Admin interface – Thanks Laura!

BluePrint CSS wrapper for BlogCFC

Quick BlueprintCSS wrapper for BlogCFC v0.1

This is a set of starter stylesheets using the BluePrintCSS framework (see http://www.blueprintcss.org/ for full usage) for BlogCFC. As always, I take no responsibility whatsoever for anything you might do with this.

Essentially all I’ve done is add the BluePrintCSS as the main starting point, and added a few classes to the layout customtag so it works under Blueprints CSS concept. Also added a few custom classes in bp_custom.css to make it a bit prettier. Add your changes to bp_custom.css – as it’s the last one to be called, it’ll override anything set in the defaults.

Tested briefly on BlogCFC 5.9.3.006

NB: This is meant as a starting point – I’ve not spent ages with elements like the pods etc. Hopefully it’ll save someone somewhere some time – the reason I’ve bothered putting it in a zip is that I have to integrate blogCFC into most of my sites (which happen to always use BluePrint);

Install instructions:

  1. Backup/Move/Rename the file at /yourblog/tags/layout.cfm
  2. Download the zip and decompress
  3. Copy layout.cfm into /yourblog/tags/
  4. Copy the four stylesheets into /yourblog/includes/
  5. Restart the blogCFC cache using /?reinit=1

Credits:

Screenshot

Download

Railo / Apache / Tomcat / Mura CMS/ SES URL Gotcha

Another catchy title 🙂

I’ve been trying for a day or two to get SES Urls working on Tomcat/Railo/Apache/Mura;

Specifically, I’m thinking of the /index.cfm/something/ style syntax. Having installed Mura, I couldn’t get any of the actual pages past the front page to work. I got an Apache 404 error.

Sean Corfield’s Blog gave me a good starting point.

As I’d installed Railo via the Beta install script, my config was a little different to some which is what gave me the headache.

Firstly, I had to get Tomcat to receive the requests, as Apache was serving me the 404s;

In my Apache httpd.conf file, I had the following at the bottom:

JkMount /*.cfm ajp13
JkMount /*.cfc ajp13
JkMount /*.do ajp13
JkMount /*.jsp ajp13
JkMount /*.cfchart ajp13
JkMountCopy all
JkLogFile /var/log/httpd/mod_jk.log

Note, the *.cfm entry – this needs changing to:

JkMount /*.cfm* ajp13

This means apache matched the correct syntax to pass the request to Tomcat.

So at that point, I was getting 404’s served by Tomcat rather than Apache; at least they were getting to the correct place.

For me, my Tomcat install lives in /opt/railo/tomcat/, so I found the web.xml file in /opt/railo/tomcat/conf/ and changed added a servlet mapping:

CFMLServlet /index.cfm/*

And then tried again with Mura. No joy – still 404’s in Tomcat; Having re-read Sean’s entry, it turns out you have to specify if you want the pattern to match a directory;

So the servlet mapping I’d put up would match /index.cfm/*, but not, as Mura users know, the default Mura behaviour of /default/index.cfm/*

Changing to:

CFMLServlet /default/index.cfm/*

Sorts it.

I know there’s a jar file supplied by Mura to fix this – but with my seemingly different Tomcat config, I could work out how to get it working properly.

Next stage is going to have to be to get the URL rewriting working to get proper URLs like /about-us/, rather than /default/index.cfm/about-us/.