@madpilot makes

Using SSH to run remote commands using PHP. A cheat guide.

I’m working on a soon-to-be-released project that needed to run commands on a Linux server. Whilst it would be possible to use something like the exec command to run it, this would mean that the user that Apache was running as would have to have permissions to run the commands, which is less than cool. I could have messed around with sudo, but even that would open up some gaping holes, as all other websites hosted on the same box could theoretically run the same commands.

As it turns out, there is a PECL project that allows you to remotely login to a server using SSH, which would actually kill a number of birds with one stone:

  1. I can sandbox the commands that get run, by setting a special user that only has access to commands that are needed (using sudo)
  2. The web app would be able to talk to multiple servers, which wouldn’t have been possible with exec alone

The flow is simple: Login to the server – I’m using a username/password pair at the moment, but only because I haven’t been able to get public key exchange working on the server yet (interestingly, it works if I call the code from the command line), run the command, then check the output and response. There was a slight issue here,  ssh2_exec returns a pointer to a stream, which needs to be read. If there is no response (some programs complete without returning anything), then the process would block indefinately. Also, if the program fails, it might not output anything to stdout, instead outputting text to stderr, AND you miss out on checking the return status code (which quite often gives you some interesting information about the status of the program).

To get around this, I wrote this really simple bash script, that runs the command on your behalf and wraps the stdout, stderr, pwd and result in an XML envelope ready for parsing. Because you will always get the envelope returned (unless the process daemonises) you won’t get the blocking problem.

#!/bin/sh
tmp_stderr=`mktemp`
output=`$* 2<$tmp_stderr`
result=$?
error=`cat $tmp_stderr`
rm $tmp_stderr

echo "<?xml version=\"1.0\" encoding=\"UTF-8\"?>"
echo "<xmlsh>"

if [ -n "$output" ]
then
    echo "  <stdout>"
    echo "    <![CDATA["
    echo $output
    echo "   ]]>"
    echo "  </stdout>"
fi

if [ -n "$error" ]
then
    echo "  <stderr>"
    echo "    <![CDATA["
    echo $error
    echo "   ]]>"
    echo "  </stderr>"
fi

echo "  <meta>"
echo "    <pwd>$PWD</pwd>"
echo "    <return>$result</return>"
echo "  </meta>"
echo "</xmlsh>"

In a nutshell, when you call the script, it runs the program supplied as an argument, then pipes the stderr out to a temporary file, and pipes stdout into a variable. It wraps this, and the current working directory and return value in XML, and prints it out. Pretty simple, but it works.

Wanted: Dead or Alive. Some PHP developers

I’m in a bit of a predicament at the moment. As many of you know, I’m back doing the web consultancy thing full time again after a 18 month hiatus. I made a couple of rules that I’m trying hard not to break:

  1. I’m not supporting PHP4
  2. I’m not supporting or modifying badly written OpenSource projects  (basically crossing out WordPress, osCommerce or phpBB jobs)

Which (maybe un-)suprisingly I get asked for a lot. As a result, I’m looking for some PHP developers that haven’t become totally jaded by years of PHP4 abuse that I can sub-contract out to. I’d say I’m looking for mid-level developers, I’ll be there to help out find out WHAT needs to be done, and HOW it should be done – I just need someone to actually do the work.

I’m pretty anal about coding standards and software practice, so things like SVN are a must. I have a development server if you need an environment to work in.

If you think you might fit the bill, and want to work with someone who’s been around the development block a few times, email me on myles@madpilot.com.au with a short CV and your rates.

Be a good netizen: Use the correct HTTP response code

Remember the good ol’ days back before dymanic websites where pages had .html extensions and when you tried to access a page that didn’t exist you got an ugly, yet reassuring 404 Not found page? The significance of this page is actually pretty important – not only does it tell the user that the page is not found but it returns a special HTTP status that tells web spiders the same thing. As web developers, sometimes we forget that humans aren’t the only ones accessing our pages, and as a result don’t use the correct HTTP response codes to denote what is going on.

What the hell is a HTTP response code?

When your web browser makes a request to a web server, the web server will return a status code as well as the web page, which tells your browser what has happened. This response is usually made up of two parts: a number (which is for any spiders or bots that might be accessing the web site) and a string (which is for humans) and back when everything was static the web server took care of everything.

Unfortunately for web developers, in this environment of database driven web sites, we often don’t have the luxury of letting the server take care of everything, so this article aims to show you that it isn’t that difficult to do HTTP responses correctly. As I will show later, this can adversely affect your ranking in your favourite search engine.

Let’s try it out

Firstly, lets see what happens when you actually make a request – you can see what is going on using another old school application: Telnet.

Open up command line or Terminal.app or terminal depending on you flavour and type the following:

telnet madpilot.com.au 80

You will get a prompt and type the following (Windows users might not see anything as the telnet client won’t echo what you type):

HEAD / HTTP/1.1

Host: madpilot.com.au

…and hit enter twice – you should see something like:

HTTP/1.1 200 OK

Server: Mongrel 1.0.1

Status: 200 OK

Cache-Control: no-cache

Content-Type: text/html; charset=utf-8

Content-Length: 4123

The first line of the response is the important bit – it tells the web browser that the response conforms to HTTP version 1.1 and more importantly the response code is 200 and the response type is OK! The 200 type is the most common response you will come across, it means that the page was found and served up correctly. Generally your web server WILL take care of this one for you. Let’s look at how you can change that status code.

I’ll use PHP as an example, because it is still the most common dynamic language – but you can do this in any language, leave a comment if you would like an example of how to do it in another dialect. It is all very simple – BEFORE you output any HTML, call the header() function as such:

header("HTTP/1.1 404 Page Not Found");

As you have probably guessed, this will tell the browser that the page it requested was not found. Why would you want to do that? Obviously if the script is being run, it has been found? Well, yes – that is true, however, when the HTTP specification was written, CMSs and dynamic product catalogues weren’t even thought about – so we need to think a little bit differently.

Let’s look at an example: Your customer has requested to see the details for product #10 by browsing to

http://www.yourcomany.com/products/view/10

Product #10 exists, so we serve up the details, but what if the customer decides to see what product #20 is? If product 20 doesn’t exist, then what should we do? One option is to print out “Product not found” which is fine for the user, but what happens if your favourite search engine tries to hit product #20? If you maintain the default action the search engine will receive a 200 OK status which makes it think that the product exists and it will index it! This just pollutes the search engines’ index and hurts your ranking, which is bad. So what we can do is serve up the 404 header from above and this let’s both the user AND the search engine know that what they have requested doesn’t exist.

So what other response codes can we use? There is a complete list here, but I’ll run through a couple of the common ones:

301 Moved Permanently: This response code means the resource that has been requested USED to live here but has now moved somewhere else and will never return. Returning this status code is extremely important if you are changing the structure of your website, as you can tell the browser where it needs to go to get the resource. More importantly, it also tells your search engine to update it’s index with the new URL. You need to supply the new URL as part of the request, so it looks something like this:

header("HTTP:1/1 301 http://www.yoursite.com/new-url");

302 Found: The 302 is actually generally used incorrectly. The most common use is to redirect a user to another page TEMPORARILY which is actually what the 303 code is for. unfortunately, not all browsers support 303 and actually expect a 302 in this case. So who are we to argue? If you are a PHP developer, you have probably used

header("Location http://www.yoursite.com/somewhere");

before – this is exactly what this does.

403 Forbidden: If you wanted to really play HTTP right, you would return this code every time someone tried to access a private URL when they weren’t logged in. It means the server knows what you are trying to do, but isn’t going to let you do it.

404 Page not found: This has been covered – basically if the resource the agent wants doesn’t exist, you serve this up.

410 Gone: This page you are looking for used to exist, but it doesn’t anymore. In fact there isn’t even a new URL, so if you are a search engine, just forget about it. Whether search engines listen to this, I’m not sure, but it can’t hurt.

500 Server Error: Something went wrong with the server. I would throw this up if there is an error that is stopping the page from loading, such as a missing database or a broken web service or similar.

Don’t forget that you can also server up content to the browser (in fact, if you don’t humans will just get a blank page), so it is recommended that you serve up a nice friendly message to your visitors explaining what happened.

So there you go – now there is no excuse for serving up errors to your users and forgetting about our automated friends. So when you are writing your next kick-arse web app, spare a thought for the visitors that aren’t so good at parsing human talk.

PHP 4 being put out to pasture

Ok, I might be a bit of a Rails zealot now, but I still have a sweet spot for PHP. For many years it was my language of choice, and even today, server hosting or legacy applications still means that I have to throw around the < ?php ?> tag.

I just read on the official PHP website, that as of the end of this year, PHP 4 will no longer be updated, bar crucial security patches. This is a big thing, as many web hosting companies still only support PHP 4 as it isn’t possible to run PHP 5 on the same apache server with out resorting to CGI or proxy work-arounds. This basically means that you have 5 months to make sure that your webiste runs on PHP 5.

The PHP website has a guide to migration, and luckily, most of the time things are fairly smooth. Were you might get in to trouble is the new object model. PHP 5 has a brand new, closer-to-real OOP model that isn’t always completely backwards compatible with it’s older sibling. For example, the __construct() method is called as a constructor, rather than function with the same same as the class as per PHP 4. PHP 5 also supports private, protected and public accessors and methods, as well as native XML and SOAP support.

At this point if, you may want to try running all of your websites as CGI scripts via the PHP5 interpreter to make sure that they can run ok, and if they do, start moving everything over to ensure a smooth transition.

Image verification for comments in WordPress 2.0

I’m sick of blog spam. It is such a pain to have to mark it all as spam (And what exactly does that function do? I’ve seen the same spam come through many times, so it doesn’t seem to learn…)

Anyways, here is my quick and dirty hack to add a image verification box on the comments page (Warning WordPress hack ahead – I’ll bother to learn the plugin system one day…)

This hack requires the following PHP libraries installed on your server: Freetype, GD and mcrypt.

Create a new file in [path_to_your_blog]/wp-image-verify.php

Insert the following code:

  1. < ?php
  2. require( dirname(__FILE__) . ‘/wp-config.php’ );
  3. $verifier = $_REQUEST[‘verifier’];
  4. $display_string = decrypt($verifier);
  5. $image_details = imagettfbbox (10, 0, ‘/usr/share/fonts/TTF/tahoma.ttf’, $display_string);
  6. $image = imagecreatetruecolor($image_details[4]$image_details[5], (0$image_details[5])(0$image_details[3]) + 1);
  7. imageantialias($image, true);
  8. $background = imagecolorallocate($image, 255, 255, 255);
  9. imagefill($image, 0, 0, $background);
  10. imagettftext($image, 10, 0, $image_details[0] + 1, 0$image_details[5], imagecolorallocate($image, 119, 119, 119), ‘/usr/share/fonts/TTF/tahoma.ttf’, $display_string);
  11. header(“Content-type: image/png”);
  12. imagepng($image);
  13. ?>

Note – you will probably (ok will) have to change the path of the font as /usr/share/fonts/TTF/tahoma.ttf is a font I copied from my windows directory.

Open [path_to_your_blog]/wp-comments-post.php and add the following lines at around line 25:

  1. $verifier = trim($_POST[‘verifier’]);
  2. $verifier_compare = decrypt($_POST[‘verifier_hash’]);

Open [path_to_your_blog]/includes/comments-functions.php and add the following lines at the top of the file:

  1. // image verifier functions – by Myles Eftos
  2. $key = “afedss”; // Enter a 5 character string
  3. $iv = “sdfesdf”; // Enter a 7 character IV
  4. function random_string($length = 6) {
  5. $str = ;
  6. for($i = 0; $i < $length; $i++) {
  7. $str .= chr(rand(ord(‘A’), ord(‘Z’)));
  8. }
  9. return $str;
  10. }
  11. function encrypt($string) {
  12. global $key;
  13. global $iv;
  14. return base64_encode(mcrypt_encrypt(MCRYPT_DES, $key, $string, MCRYPT_MODE_ECB, $iv));
  15. }
  16. function decrypt($string) {
  17. global $key;
  18. global $iv;
  19. return chop(mcrypt_decrypt(MCRYPT_DES, $key, base64_decode($string), MCRYPT_MODE_ECB, $iv));
  20. }

Replacing the $key and $iv variables with a random 5 and 7 cahracter string respectively.

Finally find the comments.php file in you theme directory – if you are using the default theme it is at [path_to_your_blog]/wp-content/themes/default and add the following lines after the url textbox (Around line 89 in the default template):

  1. < p>

  2. < input type=“text” name=“verifier” id=“verifier” size=“22” />
  3. < label< small>Enter the string you see here: < img src=“< ?php echo get_option('siteurl'); ?>/wp-image-verify.php?verifier=< ?php echo urlencode ($image_hash) ?>” alt=”” />< / small>  < / label>
  4. < input type="hidden" name="verifier_hash” value=”< ?php echo $image_hash; ?>” />
  5. < /p>

All done :)

Indenting highlighted source code with WordPress 2.0

Ok, after going away and thinking about the problem, I came up with a quick and dirty (But seemingly effective and compliant) way of doing the indenting. It does require a small modification to the GeSHi code. I’m going to use an empty inline tag (ins tag – which stands for insert – how appropriate…)

Open the geshi.php file (See my previous post about code highlighting in WordPress 2.0 if you have no idea what I’m talking about) and find the indent function. Search for all the nbsp elements and replace with:

  1. < ins class=“in” > < /ins>

Now open the insertcode.css file and the style.css (in your theme) and add:

  1. ins.in {margin-left: 15px;}

Voila! indenting. The output looks like this:

  1. < ?php
  2. echo “hello” . world();
  3. function world() {
  4. return “world”;
  5. }
  6. ?>

A few more notes: html and xml highlighting is pretty well broken unless you add a erroneous space between the angle bracket and the tag (See this example) and don’t try to use special characters. I’m currently tweaking the colour coding classes to make them look nicer too. I’ll post these when they are done.

WordPress 2.0 mod: Insert highlighted code.

I have complained before about not being able to insert source code into WordPress when using the TinyMCE Rich Text Editor. Sure, I could switch to plain text mode, but frankly I would prefer an RTE as it makes entering posts much quicker. There is a TinyMCE plugin called insertcode written by by Maxime Lardenois which does exactly what I wanted. In fact, it even uses the GeSHI cde highlighter class (written in PHP) to do code highlighting. There is something like 50 different languages which are supported – Very cool.

Unfortunately, installation was not quite as simple as I hoped. Here is what I had to do (Substitute [blog_root] with the absolute path to your wordpress install):

  1. Untar the GeSHI tarball and copy the geshi.php files and geshi directory to [blog_root]/wp-includes
  2. Unrar (yes rar) the insertcode RAR file to [blog_root]/wp-includes/js/tinymce/plugins/insertcode (Create the insertcode directory first)
  3. THe RAR file I go didn’t have a directory structure – I don’t know whether I just messed up or what, but I had to create the following directories: config, css, images, jscripts, langs, webservice
  4. Copy config.php into the config directory, insertcode.css into the css directory, insertcode.gif into the images directory, insertcode.js into the jscripts directory all the php files into the webservice directory and all of the language files (e.g. en.js) into the langs directory. You should only have editor_plugin.js editor_plugin_src.js and insertcode.htm files in the insertcode dir.
  5. Open get_highlighted_code.php and change the $geshi->set_header_type(GESHI_HEADER_PRE) to $geshi->set_header_type(GESHI_HEADER_DIV)
  6. Open [blog_root]/wp-includes/js/tinymce/tiny_mce.gzip.php and find the line $plugins = apply_filters – add ‘insertcode’ to the array
  7. Still in tiny_mce.gzip.php add ‘insertcode’ to the line that starts with $mce_buttons = apply_filters
  8. Finally add the following to lines to the initArray javascript array:

content_css : “/[blog_url_path]/wp-includes/js/tinymce/plugins/insertcode/css/insertcode.css”,

encoding : “xhtml”,

This should add a new button to your toolbar with the letter “C” into. Pressing the button should pop a dialog box in which you can cut and paste code.

To make it look nice on the front end cut and paste the contents of insertcode.css into you themes’ style.css file:

That SHOULD be it.

Sample Code
  1. < ?php
  2. echo print_message();
  3. function print_message() {
  4. return “Hello World!”;
  5. }
  6. ?>

You will notice that the indentation gets stripped – this is (I think) a WordPress thing. I’ll post a solution once I figure one out :) Also, it can be a little tempermental – you have been warned. And as always: YMMV.

Edit: I’ve fixed the indent bug: check out this post.

Cookie-less sessions in PHP

There is a fairly good chance that all but the most trivial dynamic sites will use sessions to emulate a stateful environment. Before the creation of sessions, the developer would have to manually pass all of the variables the next page needed via hidden input fields or cookies. As you can imagine, this is a bit of a security risk. Not to mention that cookies are limited to 4k each – if you have a large number of variables, this can quickly run out.

PHP, like most server languages can handle your sessions for you – but how does PHP keep track of sessions? There are two ways – using cookies and hidden fields. The former is the preferred way as it is easiest and is more secure (But not with out it’s risks. Check out the PHP session documentation for more info on the safe use of session variables). PHP writes a cookie which holds the PHPSESSID variable. It looks for this value when start_session() is called and if it exists, matches it to the stored session. But what happens when cookies aren’t available or are turned off? PHP is smart enough to work this out and will re-write all links and forms on the fly to ensure that the PHPSESSID is passed on every action that will refresh the page.

Well, almost. When I code, I prefer to postback to the same page, do some processing (such as error validation and storing of required session variables) and then use header(“Location: “) to re-direct the user. This makes coding much easier because all of the business logic for that page is kept in one file. Unfortunately, PHP isn’t smart enough to re-write these redirect, so we will have to help it out a bit.

append_sid

The following function will look at a URL (Whether it be relative or absolute) and append the SID if and only if PHP is in cookieless mode. I couldn’t find a PHP function that will tell you whether cookies are working, however after a bit of thought it was quite easy. PHP has a super-global called $_COOKIES. In this super-global, the PHPSESSID variable is referrenced. By checking whether it exists (After a call to session_start) we can work out whether we are in cookie or cookie-less mode:


function append_sid($link) {

    if(session_id() !== NULL && !isset($_COOKIE['PHPSESSID'])) {

        if(strpos($link, "?") === FALSE) {

            return $link . "?PHPSESSID=" . session_id();

        } else {

            return $link . "&PHPSESSID=" . session_id();

        }

    } else {

        return $link;

    }

}

Basically what happens, if the function checks to see if the session has been started – if it has, it then checks if the cookie entry exists. If it does, it just returns the link unchanged, otherwise it appends the session id. The inner most if statement checks for a ‘?’. If it exists, we can assume that the link already has some get parameters, so we will append the PHPSESSID using an ‘&’. Otherwise, just append it with a ‘?’.

JavaScript

Don’t forget that if you are re-directing client using javascript or using sessions when making AJAX calls you need to pass the session id as well. The best way to do that is client side, especically if you are generating parameters on the fly. The same theory applies, except we will use JavaScript this time:


function append_sid(link) {

    < ?php if(session_id() !== NULL && !isset($_COOKIE['PHPSESSID'])) { ?>

        var session_id = '< ?php echo session_id ?>';

            if(link.indexOf('?') == -1) {

                return link + '?PHPSESSID=' + session_id;

            } else {

                return link + '&PHPSESSID=' + session_id;

           }

    < ?php } else { ?>

        return link;

    < ?php } ?>

}

The reason there is PHP code interleaved (I know. I hate it too) is because we need to work out whether the session has actually been started and we need the session_id. However, we don’t want to display the session id UNLESS there is a requirement for it as it can be a security risk.

A caveat – passing session id by URL requires the session.session.use_trans_sid tio be set to 1. Most shared hosts turn this off, but you can usually override it by putting:

php_value session.use_trans_sid = 1 in the .htaccess file. You can also try the function:

ini_set('session.use_trans_sid', '1')

On every PHP page.

Now, hopefully, you won’t be cutting off those users that don’t use cookies!

PHP 5 and MVC

I quick entry today, as I really should be working.

I came across a PHP MVC (Model/View/Controller) framework for PHP 5 – You can think of it as the Rails bit of Ruby on Rails. It is this sort of stuff that will continue to push PHP into the spotlight and allow it to compete with the big boys…

It is called Agavi PHP MVC Framework

Design Patterns in PHP

I was going thorugh the posts from my old, now-defunct blog, seeing if there was anything I could bring over here — it is amazing how much can change in a year. There was an article I wrote about over use of cool techniques. In that article, I made mention to some new fangled technique called “Design Patterns”. At that point, I had no idea what they were and frankly couldn’t care.

Well, after being forced to look at them more closely for a uni assignment, I’m kind of a little hooked.

Design patterns are abstract solutions to common problems. Huh? Yeah, that is what I said. Many programmers strive towards code re-use. Design Patterns encourage thought re-use. Why re-invent the wheel? And because they are abstract (i.e. no code), they can be “ported” to different languages easily.

The Gang of Four introduced 23 of the things. They have put the challenge out for the discovery of more, and they haven’t been too successful as it is believed that almost all problems can be broken down to fit a composition of these rules.

To implement many of these design patterns correctly, you really need OOP features such as Abstraction and Interfacing. As I have pointed out many times, PHP4 doesn’t have these. PHP5 does. However! if you think about it, you can still use design patterns in PHP4 — you just need to be a little bit careful.

I won’t go through ever design pattern just now, but, I’ll outline one, and maybe add to them in the future. [By the way — an awesome book to use for getting your head around design patterns is “Head First Design Patterns” — look it up on Amazon]

The Stategy Pattern

Programmers often get into the habit of extending classes to change functionality — maybe because it is the easiest OOP function to understand. It can lead to problems though when you need to change implementations in a base class. I’ll use an example from a system I was writing the other day: the ubiquitous PHP email sender. The job I was doing required two types of email to be sent — a run-of-the-mill text email and a text email with an attachment.

Because I’m always trying to build my code libraries up, I decided to create set of classes that will do the job. Now, pre-design patterns, something like this may have happened:

Possible Email Class design using extension

Nothing wrong so far, but what happens if we want to add a HTML email? We could probably add another class below the attachments, because the HTML component is an attachment I guess, so we add the HTML file as one of the attachments at design time. This isn’t a great solution though, because one of the necessary bits to a HTML email is the HTML content, and you could theoretically remove it. How about we add another variable specifically for the HTML copy? That would also work, but what if the client decided that they wanted to select between HTML and text emails dynamically at run time? You would end up with something like this code wise:

switch($mailType) {
case “text”:
$email->setBody($text);
break;
case “html”:
$email->setHTMLBody($htmlText);
break;
}

(This is a bit of a contrived example, so run with me here)

now what happens if you want to add another type of email? You have to modify the case statement and test the whole thing all over again. Everyone hates testing – especially code that you know used to work that you had to change. Two keywords: CHANGE and TESTING. Minimise both and we as programmers are happy.

Now my solution was to create a class that has everything an email needs – To, From , Subject and a repository for headers (Plus a few other bits and pieces, which I won’t mention to reduce clutter). It also has a function called send(). Wow, ground breaking. Where the design gets a little strange, is there is another variable, called $emailType. This variable stores reference to a class of type EmailType (Well it pretends to, PHP4 doesn’t support interfaces). So, any class that implements EmailType can be stored in that variable. One of the abstract classes (again, let’s pretend it is abstract, PHP4 won’t understand) is the createMessage() function. This is where the magic occurs…

Each class that implements that interface know exaclty how it message needs to be contructed. The base email class doesn’t care – as long as it gets a string to tack on to the email it is happy. The creation of messages is de-coupled, meaning you can create a new email class without changing any exisiting code (As long as it implements the interface correctly).

This is an implementation of the strategy pattern!

Formal Description [Ganf of Four]: The strategy pattern defines a family of algorithms, encapsulates each one, and makes them interchangable. Strategy lets the algorithm vary independently from clients that use it.

Which is what we just did… More on design patterns later.