Daniel Rench

programming, etc.

previous : contact : linkedin : code : links : pictures : facebook : twitter

Git on GoDaddy

"Could you help me with my web site?"
"Where's it hosted?"
"GoDaddy! Here's the FTP login."

Don't get depressed. It's possible to make even GoDaddy's entry-level Linux shared hosting tolerable by enabling ssh and installing git, even though they seem to go out of their way to make it difficult.

The first thing to do is activate ssh for the account. It's not on by default, and you have to jump through some hoops to activate it. Log in to the GoDaddy hosting control panel for the domain. Currently, that means log in at the top of GoDaddy's home page, and then click My Account. From this page, find the My Products section and click on Web Hosting. You should soon see a list of hosted domains, with a LAUNCH button for each. Click the LAUNCH button for the domain you're working with and you will soon arrive at the "Hosting Dashboard". Locate the "Settings" tab and find the section labeled "SSH". Jump through the hoops to enable it (I needed to provide a phone number) and wait.

Take a break. It may be 10 minutes before ssh is actually working.

Remember that FTP login information? That username and password will work for ssh and sftp. You'll need to use it once, when you log in to put your ssh public key in ~/.ssh/authorized_keys. I'm going to assume you're familiar with ssh keys and won't go into how to generate them, manage them, work with ~/.ssh/config, ssh-agent, etc.

There is one twist: you need to add a command= section to the line containing your public key so it looks something like this (all on one line):

command="if [[ \"x${SSH_ORIGINAL_COMMAND}x\" == \"xx\" ]]; then /bin/bash; else source ~/.bashrc; eval \"${SSH_ORIGINAL_COMMAND}\"; fi; " ssh-rsa AAAAAAA+YOUR+KEY+GOES+HERE...

You need this for your .bashrc to have any effect on non-interactive shells (as you do when using git remotely). Your account probably doesn't even have a .bashrc yet, but it will soon.

So yes, you'll be using bash, the MSIE of shells, and probably a fairly old version as well. Considering the crippled environment (they even took which away from you, to make it even harder to know what you're missing) you're going to want to spend as little time in this shell as possible anyway.

Now let's get git.

Find out what Linux distribution GoDaddy gave you. uname -a says I'm on an i386 machine, and /etc/redhat-release says I'm on CentOS 5.5. If you're on something else (like a Debian derivative), you're on your own. Sorry!

Point your browser at RepoForge's collection of git RPMs and download the most up-to-date version of git available for your account's OS version and architecture. In my case, that was git-1.7.6.4-1.el5.rf.i386.rpm. GoDaddy may have given you wget so either use that to grab it, or fetch the package some other way and sftp it to your $HOME directory on the GoDaddy server.

Time to unpack. Fortunately, GoDaddy gives us rpm2cpio and cpio. I'm going to put things under ~/opt/:

% mkdir ~/opt && cd ~/opt
% rpm2cpio ~/git-1.7.6.4-1.el5.rf.i386.rpm | cpio -id
% ~/opt/usr/bin/git --version
git version 1.7.6.4

Hooray. We're almost there. Add this to your ~/.bashrc (or create one; this account I used didn't have one):

PATH=$PATH:$HOME/opt/usr/bin
export PATH

GIT_EXEC_PATH=$HOME/opt/usr/libexec/git-core
export GIT_EXEC_PATH

We need to set GIT_EXEC_PATH, otherwise git (the version I installed, anyway) will expect to find its 'core programs' under /usr/libexec/git-core.

Log out of this ssh session and log back in. We need to know your new .bashrc is working on login.

If you see something like bash: git: command not found, your .bashrc

Let git know where to find its templates:

% git config --global init.templatedir ~/opt/usr/share/git-core/templates

Log back in with ssh to verify that git is working:

% mkdir ~/testrepo
% cd ~/testrepo
% echo 'This is a test' > README.txt
% git init
Initialized empty Git repository in /home/content/XX/YYYYYYY/testrepo/.git/
% git add README.txt
% git commit -m 'initial version'
[master (root-commit) 5214f3b] initial version
 1 files changed, 1 insertions(+), 0 deletions(-)
 create mode 100644 README.txt

Open a new shell locally and verify that you can work with git remotely over ssh:

% git clone godaddy-host.example.com:~/testrepo
Cloning into testrepo...
remote: Counting objects: 3, done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Receiving objects: 100% (3/3), done.

Don't you feel relieved now? If you're like me, you're going to want to use the rpm2cpio technique to install other missing essentials like rsync and make (to install CPAN modules, which could be, and may become, a whole new posting).

phemca, a JavaScript-to-PHP5 compiler (written in Ruby)

Why am I writing a JavaScript-to-PHP5 compliler (in Ruby)?

PHP is everywhere and is inescapable these days.

A typical hosting plan includes PHP. Maybe not PHP 5.3 yet, but definitely some version. And "regular people" like PHP, or more accurately, they like stuff written in PHP: Wordpress, Drupal, Magento, SugarCRM, and just about every other common CMS, shopping cart, and bulletin board system.

Irregular people (programmers) typically aren't as keen on PHP. Programmers may not agree on many things, but they do tend to think JavaScript is not too bad, and that's high praise. It's everybody's 2nd favorite language.

I don't know about you, but when given a task to, say, write a plugin for some PHP app, I would love to write it in a language I enjoy using. Sure, I could always throw out the entire codebase and start over in node.js or something, but I don't have time for that.

Or maybe I do, but I doubt my clients would pay me to do that. And even if they did, they won't necessarily be happy clients when they find out that their "Wordpress" site won't let them use any Wordpress plugins, because they're not actually running Wordpress anymore, but some incompatible custom blog engine running on a strange new platform.

Ideally, this compiler I'm calling phecma could be part of a piecemeal transition to a better place. No need to throw out the entire codebase and platform. Boil that PHP frog slowly. Eventually, it's all ECMAScript. When that happens, this is like the Wordpress Singularity or something.

Intrigued? Phecma is on Github.

Backing up MySQL: The Last Resort Method

One of the first things I do when taking on maintenance for a web site is to do my own backups. This usually means database snapshots, and that database is usually MySQL. Unfortunately I am rarely allowed a shell login to the server, and sometimes not allowed to connect to MySQL from outside the host's network either. But one thing I can always expect is access to PhpMyAdmin, the virtually-standard web frontend for MySQL administration.

That's fine, but I need my backups automated. I found a Python program that screen-scraped PhpMyAdmin, but I ran into a big problem: the PhpMyAdmin installation for one site had a robots.txt file at its root that excluded everything. Python's urllib2 honored the robots.txt rules and I was not finding anything in its documentation on how to override it. It was time to fall back to something that would bend to my will: Perl 5 with WWW::Mechanize.

I gave it the imaginative name phpmyadmin-backup and added it to my 'futilities' collection on github. It works like this:


% phpmyadmin-backup \
    --url=http://URL_of_your_PhpMyAdmin/ \
    --dbname=database \
    --username=username \
    --password=password   > BACKUP.sql.gz

If you're backing up a Wordpress site, point phpmyadmin-backup at a copy of the site's wp-config.php file and it will derive the dbname, username, and password from it:


% phpmyadmin-backup \
    --url=http://URL_of_your_PhpMyAdmin/ \
    --wp_conf=/path/to/wp-config.php       > BACKUP.sql.gz

Your prerequisites (system requirements, if you will) are just Perl 5 with WWW::Mechanize. Also, php must be in your $PATH if you want to use the --wp_conf option.

Enjoy, but remember: this is a last resort for cases where mysqldump isn't an option. I genuinely hope you never have to use this program.

coordinate.js adds interesting properties to boring HTML tables

Did you ever wish you could address HTML table cells by X and Y coordinates?

var table = document.getElementsByTagName('table')[0];
coordinate(table);

// The origin (0,0) is the top, leftmost cell
table.coordinates[3][2].innerHTML = '(3,2)';

Did you ever wish you could, given a table cell, easily address its neighbors?

var table = document.getElementsByTagName('table')[0];
coordinate(table);

var cell = table.coordinates[3][2];

// 'edge' cells have 1 undefined direction
// corner cells have 2 undefined directions
// a cell in a 1x1 table would have all 4 directions undefined
// (but why would you do that?)

if (cell.north) cell.north.innerHTML = 'up';
if (cell.south) cell.south.innerHTML = 'down';
if (cell.west)  cell.west.innerHTML  = 'left';
if (cell.east)  cell.east.innerHTML  = 'right';

Did you wish you could, given a table cell, know its X and Y coordinates?

var table = document.getElementsByTagName('table')[0];
coordinate(table);

var cells = table.getElementsByTagName('td');

for (var i=0,td; td=cells[i]; ++i)
    td.onclick = function () {
        console.log('You just clicked on the cell at ' + this.x + ',' + this.y);
    };

Try the demo!

Of course it's on github

Shebang: What?


% /usr/local/php5/bin/php --version
PHP 5.2.6 (cli) (built: May 11 2008 13:09:39) 
Copyright (c) 1997-2008 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2008 Zend Technologies
    with Zend Extension Manager v1.2.2, Copyright (c) 2003-2007, by Zend Technologies
    with Zend Optimizer v3.3.3, Copyright (c) 1998-2007, by Zend Technologies

% echo '<?= phpversion() ?>' | /usr/local/php5/bin/php
5.2.6

% printf '#!/usr/local/php5/bin/php\n<?= phpversion() ?>' > version
% chmod +x version
% ./version
5.2.6

% mv version x.php
% ./x.php
4.4.9

In case you're wondering, this machine is running Linux, not Windows.

November 6, 2009 update: I asked and got a quick answer on serverfault: somebody changed /proc/sys/fs/binfmt_misc/php.

modiff: The Perl Module Differ

Here's something else I wrote (like acuff) for working on Apache/Perl installations, particularly unfamiliar ones: modiff, the Perl module differ.

Say you need to update a certain Perl module. You check out the latest version, make your edits, "make install", and everything's great. Usually. Only this time you've just overwritten a hack that guy who got laid off never committed 6 months ago, and now everything's broken. Even rolling back to the previous svn revision can't save you now.

You should have run modiff first:

% cd ~/project/Really-Important-Module && svn update && modiff
U    lib/Really/Important/Module.pm
Updated to revision 32767.
*** /usr/local/lib/perl5/site_perl/5.8.7/Really/Important/Module.pm   Tue Apr  1 16:59:59 2008
--- lib/Really/Important/Module.pm    Wed Jan  7 20:35:37 2009
***************
*** 2,9 ****

  sub init {
        my $self = shift;
!       # $self->{conf} = parse_config('/etc/x.conf');
!       # This always bombs. Comment out until I figure out why
        return 1;
  }

--- 2,8 ----

  sub init {
        my $self = shift;
!       $self->{conf} = parse_config('/etc/x.conf');
        return 1;
  }

I first wrote modiff in Ruby but I eventually needed to use it on a server that did not have Ruby, and you probably will too. The version here is the sensible Perl rewrite.

Note that modiff works only if your module's source is in CPAN-style where *.pm files are under lib/. For nonstandard directory layouts or detecting changes to other files like *.pod or anything XS-related, you're on your own.

October 5, 2009 update: modiff is now part of my "futilities" collection on github.

acuff: Apache Configuration File Finder

If you've ever worked on an unfamiliar Apache installation you know how it is trying to follow all the includes (which may in turn include more files) to track down the source of a mysterious rewrite rule or LocationMatch. So I wrote acuff. It searches for an Apache executable in your $PATH and typical locations like /usr/local/apache2/bin and spits out its config filenames (with full paths) to standard output. My typical usage goes something like this:

% acuff | xargs grep -il '<location /logout'
/usr/local/etc/apache22/conf/app.conf

It's in Perl, and anything running Apache is bound to have Perl on it. It uses no extra CPAN modules but it does need Perl 5 (this being 2008 it's completely reasonable to ignore Perl 4 and earlier, I hope). Enjoy.

October 5, 2009 update: acuff is now part of my "futilities" collection on github.

Javascript (and Perl and Ruby) Bloom Filters

I've never seen a Javascript Bloom filter implementation so I had to write one, and then had to port it to Perl and Ruby. You'll understand why soon if you don't already.

Quick aside: more than half of "my" bloomfilter.js is somebody else's BSD-licensed Javascript MD5 library but you could pretty easily replace it with some other hash library if you felt the compulsion.

I don't think this one would be practical for really large input sets (like a spell check dictionary built from /usr/dict/words), but I'm sure you'll think of something if you work at it. Get moving on that.

To get your brain in gear, here's a small demonstration.

First, I put the text of a really excellent Wire song (I won't say which), one word per line (minus punctuation) in a file. Then using the Ruby version I ran the file through this:

require 'bloomfilter'

bf = BloomFilter.new({ 'bits' => 0xff, 'salts' => %w(z A) })

STDIN.each_line { |l| l.chomp!; bf.add(l.downcase) }

print(b.to_json)

Or maybe I used the Perl one and ran it through this:

require 'bloomfilter.pl'; # what is this, 1994?

my $bf = BloomFilter->new({ bits => 0xff, salts => [qw(z A)] });

while(<>) { chomp; $bf->add(lc $_); }
print $bf->as_JSON();

Doesn't matter. Both give the same output (though I changed the formatting to make it a little easier to read):

{
 "buckets":[1509645767,990260715,1062483727,150062916,551224088,354022058,1818090233,1463314593,110],
 "bucketsize":31,
 "bits":255,
 "salts":["z","A"]
};

Type some Wire lyrics into the box below and smell the magic:

 

The filter check code runs 'onchange' for that text box. Just type (or paste in) some text, then click outside the box and you should get some feedback.

I encourage you to view the source of this page to see what's really going on but here's a quick example of the Javascript interface for the impatient:

// choose an appropriate number of bits and salts
// (don't ask me; you can figure it out)

var bf = new BloomFilter({ bits: 0xffff, salts: ['x','y','z','!'] });

bf.add('one');
bf.add('two');
bf.add('three');

bf.test('one'); // will definitely return true
bf.test('two'); // will definitely return true
bf.test('three'); // will definitely return true

bf.test('eins'); // will probably return false
bf.test('zwei'); // will probably return false
bf.test('drei'); // will probably return false

If you find bugs in this, please let me know. Also I'd really like to hear from anyone who finds other cool uses for this (aside from Wire lyric guessing games). And just to get this out of the way, I have no idea if this code can interoperate with the Bloom::Filter module on the CPAN but I should probably find out.

Hash::Server: a (mostly) memcached-compatible network interface to Perl hashes

You can download Hash::Server here until I get it on the CPAN (after a decade with Perl as my primary programming language I've finally signed up for a PAUSE account, so it could happen). Maybe you'll find this useful.

April 29, 2009 update: this code is now on github.



SYNOPSIS

        use Hash::Server ();
        use DB_File (); # for example
        tie(my %db, 'DB_File', './hash.db') or die $!;
        Hash::Server->new({ hash => \%db, localport => 11211 })->Bind();


DESCRIPTION

While memcached's original purpose was as a networked Memoize, it's also a simple way to distribute key/value hashes. But the official memcached (as the name implies) stores its data in memory, and is temporary. Given there are so many memcached clients for so many languages, it seemed a shame that there were no other ``backing stores'' available.

This module allows you to distribute any Perl hash structure you like with the memcached protocol. Typical use would probably involve a tied hash (as in the DB_File example above); if you want to serve a straight untied hash, you might as well use the real memcached, especially given the limitations of this module...


CAVEATS

This module is not a complete implementation of the memcached protocol.

  • It ignores any expiration times you provide; values are always assumed permanent.
  • It ignores any incoming ``flag'' value you provide when storing, and returns 0 by (default) when getting. Changing $Hash::Server::DEFAULT_FLAG to another value may help you on the client-side to differentiate a nonexistent key due to a server outage, or a nonexistent key because the key just does not exist.
  • With standard memcached (flags field aside, which you can't get with the Perl clients anyway), there is no difference between a nonexistent key and a down server. For pure caching, that's fine, but your application may need to know the difference. You can set $Hash::Server::UNDEFINED_KEY to some value that makes sense to you, such as 0 or q{} (empty string) and test for that, knowing that if your client returns 'undef', it means it didn't get that answer from the server.
  • The ``stats'' command is unimplemented.


SEE ALSO

http://code.sixapart.com/svn/memcached/trunk/server/doc/protocol.txt


AUTHOR

Daniel Rench, <citric@cubicone.tmetic.com>


COPYRIGHT AND LICENSE

Copyright (C) 2007 by Daniel Rench

This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself, either Perl version 5.8.7 or, at your option, any later version of Perl 5 you may have available.

It might help somebody: Flash 9 with Firefox on a Mac

When I install Firefox on a Mac, I always sudo chown -R 0:0 /Applications/Firefox.app after. What, you don't? But when trying to install Flash 9, I got this:
1008:5-5000 Access Denied Error
You do not have enough access privileges for this installation.
So I tried running the Flash install as root. Same error. Then I did this: sudo chown -R drench /Applications/Firefox.app and ran the install again, which worked, and it made me sad.

older things