Install Node on the Nagios machine (My Nagios XI VM running CentOS… for other platforms see nodejs.org)

wget http://nodejs.tchol.org/repocfg/el/nodejs-stable-release.noarch.rpm
sudo yum localinstall --nogpgcheck nodejs-stable-release.noarch.rpm
sudo yum install nodejs-compat-symlinks npm
rm nodejs-stable-release.noarch.rpm

Install the “commander” package by TJ Holowaychuk for easier option processing.

$ sudo HTTP_PROXY=$HTTP_PROXY npm install -g commander
npm http GET https://registry.npmjs.org/commander
npm http 200 https://registry.npmjs.org/commander
npm http GET https://registry.npmjs.org/commander/-/commander-1.0.4.tgz
npm http GET https://registry.npmjs.org/keypress
npm http 200 https://registry.npmjs.org/keypress
npm http GET https://registry.npmjs.org/keypress/-/keypress-0.1.0.tgz
commander@1.0.4 /usr/lib/node_modules/commander
└── keypress@0.1.0

Install PhantomJS and CasperJS

Again on the NagiosXI VM:

# Get PhantomJS and install it in /opt/ with a symlink in /usr/local/bin/
#  http://phantomjs.org/download.html
wget http://phantomjs.googlecode.com/files/phantomjs-1.6.1-linux-x86_64-dynamic.tar.bz2
bunzip2 phantomjs-1.6.1-linux-x86_64-dynamic.tar.bz2
tar xf phantomjs-1.6.1-linux-x86_64-dynamic.tar
 
sudo mv phantomjs-1.6.1-linux-x86_64-dynamic /opt/
sudo ln -s /opt/phantomjs-1.6.1-linux-x86_64-dynamic/ /opt/phantomjs
sudo ln -s /opt/phantomjs/bin/phantomjs /usr/local/bin/
 
which phantomjs      # should return /usr/local/bin/phantomjs
phantomjs --version   # should return `1.6.1` with no errors
 
# Check out and install the latest CasperJS with Git
#   http://casperjs.org/installation.html
sudo yum install git -y
 
git clone https://github.com/n1k0/casperjs.git 
 
sudo mv casperjs/ /opt/casperjs
sudo ln -s /opt/casperjs/bin/casperjs  /usr/local/bin/
 
 
which casperjs    # should return /usr/local/bin/casperjs
casperjs --version  # should return `1.0.0-RC1`

– Ryan Dahl, creator of Nodejs.

I’ve been using Ethan Schoonover’s obsessively comfortable “Solarized” color scheme for a while now, but I’ve become annoyed how many keys I have to mash to switch from light to dark. So I set a global variable to track the current state and mapped Ctrl-c, d (which I at least wasn’t using) to a function to toggle between the two states.

(when window-system
 
  ;; ColorTheme - see http://www.emacswiki.org/cgi-bin/wiki/ColorTheme
  ;; and see gallery at http://www.cs.cmu.edu/~maverick/GNUEmacsColorThemeTest/
  ;; try also M-x color-theme-select
 
  (message "loading color-theme")
  (add-to-list 'load-path "~/.emacs.d/color-theme-6.6.0")
 
  (require 'color-theme)
 
  ;; Ethan Schoonover's Solarized theme from
  ;; https://github.com/sellout/emacs-color-theme-solarized
  (load-file "~/.emacs.d/emacs-colors-solarized/color-theme-solarized.el")
 
  ;; Set initial theme to "dark"
  (setq dark-or-light 'dark)
  (color-theme-solarized dark-or-light)
 
  ;; Shortcut to toggle between light and dark
  (global-set-key (kbd "C-c d")
                    (lambda ()
                      (interactive)
		      (if (eq dark-or-light 'light)
			  (setq dark-or-light 'dark)
			  (setq dark-or-light 'light)
			)
                      (color-theme-solarized dark-or-light)))
  )

And ooh, I just found this awesome crash course on Elisp by Steve Yegge which I will certainly consult next time I have a reason to write my own (i.e., I can’t paste it from some random Internet).

And then after it was all over I finally found some instructions for doing it which would’ve been easy enough to paste. Ah well, I’ve learned something.

1. Upgrade PEAR

1. Download http://pear.php.net/go-pear.phar to C:\xampp\php\go-pear.phar

2. Run this in cmd.exe (cygwin prompts hosed somehow), taking all defaults

c:\xampp\php>go-pear.bat

1. Yay.

c:\xampp\php>pear version
PEAR Version: 1.9.4
PHP Version: 5.3.5
Zend Engine Version: 2.3.0
Running on: Windows NT FAI1046162 6.1 build 7600 (Unknow Windows version Enterpr
ise Edition) i586

2. Install/upgrade PHPUnit

Now we can use the cygwin shell. Not sure all these channels are needed, I did this out of order.

cd /cygdrive/c/xampp/php
pear update-channels
pear channel-discover components.ez.no
pear channel-discover pear.symfony-project.com
pear channel-discover pear.phpunit.de
 
pear install --alldeps phpunit/PHPUnit

3. Use it

XAMPP and/or PHPUnit come with a wrapper called phpunit.bat, which has now been upgraded, but you may need to set your PHPBIN environment var. Also I already have c:\xampp\php in my $PATH.

export PHPBIN=c:/xampp/php/php.exe
 
cd /path/to/mysf2project
phpunit.bat -c app

Now PHPUnit works. Make tests and make them work!

Sorting

Not immediately obvious, however, was how to sort them. This did the trick for my case (I have a Sequence field manually re-calculable through a jQueryUI sortable widget. Also note the cool inline anonymous function syntax available since PHP 5.3. Incidentally, I’m not sure the terms lambda or closure are helpful because they’re not quite like Lisp lambdas or JavaScript closures.

// Re-sort them by Sequence, numerically
$this->collSegments->uasort(function($a, $b) {
    return $a->getSequence() - $b->getSequence();
});

// Re-sort them as strings, case-insensitively.
$this->collSegments->uasort(function($a, $b) {
    return strnatcasecmp($a->__toString(), $b->__toString();
});

Deleting

Thanks to the PropelArrayCollection API Documentation

  /*
   * Remove the provided Segment object.
   *
   * @param Segment $s
   * @return Segment $s that was deleted.
   */
  public function deleteSegment(Segment $s) {
      $s->delete();
      $key = $this->collSegments->search($s);
      $ret = $this->collSegments->remove($key);
      return $s;
  }

So elsewhere,

	      $this->deleteSegment($s);

On my Snow Leopard machine this kept happening.

• The “open” indicator (glowing silver ball under the app icon) in the Dock was flaky, only showing for a few apps though more were running.
• The task switcher (which you see when you option-tab/alt-tab) didn’t show all running apps. That RUINS it for me. I always use option-tab.
• I hate rebooting.

Workaround–Restart the Dock process.

Open Terminal† and use one command:

killall -HUP Dock

Don’t be afraid of “killall”. HUP means “Hang Up” and is the normal way of telling something to relaunch.

Thanks to AcmeTech’s old post.

† Terminal is located in the Utilities folder in the Applications folder. Or the quick Spotlight way is command-space, Terminal.

Monitoring tools are to the sysadmin what testing tools are to the developer.

Recently I realized that there need to be more ways to bring both toolsets together. Here’s one, tying the Nagios monitoring toolset to anything that emits the popular Test Anything Protocol.

The sysadmin community loves monitoring tools. Partly because of Tom Limoncelli’s time management book, I’ve been using Nagios for several years now to monitor all sorts of IT hardware and software. It includes features for scheduling check scripts and contact rules (page me during weekdays, page someone else at night), and dashboard views for the visually oriented (curiously unnecessary once you have alerting configured well). I’ve found it takes investment of time and attention to configure all the checks you want, but it’s worth it. Because of its open-source, extensible nature, Nagios is especially good for monitoring weird things that other enterprise monitoring systems aren’t even aware of. There are a lot of books about Nagios available, and I’d recommend them for anyone new to Nagios. I found James Trumbull’s book published by APress particularly useful, though it looks like it might be due for a new edition by now.

The developer community has become increasingly interested in software quality control and test driven development (TDD). I first became interested because of the Perl community’s emphasis on Kwalitee testing ever so long ago. There are even more good books and online resources about testing than there are about monitoring. I have been noticing everywhere though, that Perl’s simple and ancient Test Anything Protocol has become somewhat of a standard, with testing tools in many languages (including JavaScript, PHP, Python, Ruby, Java, C, C++, C#/VB/.NET and database-specific languages) producing it. Everything from low-level unit tests in C to frontend cross-browser functional testing tools like Selenium. And TAP is so simple, it’s easy to wrap a TAP harness around other test frameworks.

One strategy–from the security world–on which sysadmins and developers both agree is enumerating goodness (take note, “antivirus” is Doing It Wrong). Basically, it’s too hard to guess all the myriad ways your technology might fail and then write monitoring or test scripts for them. Instead, focus your monitoring and testing on covering the important functionality of your product.

But at what level of detail should we monitor? When the monitoring system wakes you up at 2 am, which do you want it to say?

1. CRITICAL. No users can log in to http://myapp/login.
2. CRITICAL. The app can’t contact the database.
3. CRITICAL. The database server is down/unpingable.

My preference is “4. All of the above”, because the user-facing effect is sometimes hard to guess from the sysadmin-facing event (and vice versa). Nagios is good at #3 and maybe #2. #1 is more in the realm of functional testing, and your app may already have functional test scripts that could provide this sort of information.

check_tap.pl

A Nagios plugin for consuming Test Anything Protocol. Basically it combines Test::Harness with Nagios::Plugin.

Read, download, or fork check_tap.pl from the Github Gist.

Plugin Documentation (more or less straight from check_tap.pl --help)

check_tap.pl [ -v|--verbose ] [-t <timeout>]
[ -c|--critical=<critical threshold> ]
[ -w|--warning=<warning threshold> ]
[ -s|--script = '</full/path/to/test.t>' ] (Required. multiple OK)
[ -e|--exec = '/full/path/to/runnable ARGS'
[ -l|--lib = '/path/to/perl/libs' ] (Multiple OK)

    check_tap.pl -s /full/path/to/testfoo.pl

    check_tap.pl -s /full/path/to/testfoo.pl -c 2

    check_tap.pl -e '/usr/bin/ruby -w'  -s /full/path/to/testfoo.r

    check_tap.pl -e '/usr/bin/curl -sk' -s 'http://url/to/mytest.php'

    check_tap.pl -e '/usr/bin/cat' -s '/path/to/testoutput.tap'

Installation and Nagios configuration

Save check_tap.pl anywhere that makes sense and the nagios user can access. The default plugin location depends on your distribution. For RHEL and ilk, plugins are in /usr/lib/nagios/plugins/.

In misccommands.cfg:

# runs Perl test script given in $ARG1$
define command {
        command_name    check_tap
        command_line    /usr/lib/nagios/plugins/check_tap.pl -v \
               -s $ARG1$ -w $ARG2$ -c $ARG3$
}
 
# gets TAP output from the full URL in $ARG1$
define command {
        command_name    check_tap_remote_url
        command_line    /usr/lib/nagios/plugins/check_tap.pl -v \
               -e '/usr/bin/curl -sk' -s $ARG1$ \
               -w $ARG2$ -c $ARG3$
}
 
# gets TAP output from the relative URL in $ARG1$ at host $HOSTADDRESS$
define command {
        command_name    check_tap_remote_host
        command_line    /usr/lib/nagios/plugins/check_tap.pl -v \
              -e '/usr/bin/curl -sk' \
              -s http://$HOSTADDRESS$$ARG1$ \
              -w $ARG2$ -c $ARG3$
}

You can also use NRPE for *nix or NSCLient++/nscp or NC_Net for Windows (you’ll also need Perl, like Strawberry Perl) to run check_tap.pl on a machine other than the Nagios server. I’ll leave that as an exercise for the reader, or maybe a future blog post.

In services.cfg:

define service {
  use                  generic-service
  check_command        check_tap!/path/to/my/test.t!0!0
  service_description  Local test script on Nagios server
}
 
define service {
  use                  generic-service
  check_command        check_tap_remote_url!http://webhost99/nagios/mytest.php!0!0
  service_description  Remote test script accessible by URL
}
 
define service {
  use                  generic-service
  host_name            webhost1, webhost2
  check_command        check_tap_remote_host!/nagios/mytest.php!0!0
  service_description  Remote test script
}

Host-based security for URL-accessible test scripts

It would be good to limit access to the URL-accessible test scripts to your Nagios server and development/admin network. So for Apache, .htaccess or httpd.conf:

Order deny,allow
Deny from all
Allow from nagioshost.example.com
Allow from developers.example.com

For IIS you can click on stuff for the same result or use the deny element in your web.config file. For other web servers, RTFM.

Doughnuts for Developers

Wow! Now I can use Nagios’s monitoring, alerting, performance logging, acknowledging, scheduling, dashboarding, and thresholding features for my unit and functional tests! I can run them with different rules in build and production! I can detect random and senseless acts of system administration! I can detect new bugs as soon as I write them! I can hook Nagios up to the sprinkler system to put a damper on that hotheaded programmer in the next cube! I can write frontend functional tests with Selenium or AutoIt or AppleScript and have Nagios alert (the sysadmins of course) when any of my app’s core functionality breaks!!!

There are plenty of other test harnesses out there, but Nagios has some big advantages, especially for monitoring production systems.

Shortcuts for Sysadmins

Two last things which are more good news for sysadmins:

Custom test scripts for things Nagios won’t easily check

Writing your own Nagios plugins is a little bit hard, even with the example plugin I worked on ever so long ago. Writing TAP test scripts, on the other hand, is easy! So, after you’ve set up Nagios monitoring for all the low-hanging fruit like hardware utilization and network availability, write a test script for some of the harder to monitor signs of enumerable goodness. Here’s an example:

#!/usr/bin/perl
use warnings; use strict;
 
# This allows the script to run as a CGI or on the command line.
# We have to output the header in a BEGIN block or Test::Simple will
#   output the plan too soon.
BEGIN {
    if ($ENV{REQUEST_METHOD}) {
        use CGI qw(header);
        print header("text/plain");
    }
}
 
use Test::Simple tests=>4;
use Test::File;
 
# These use Test::File to test file permissions and size.
# See http://search.cpan.org/perldoc?Test::File
file_writeable_ok "path/to/cache";  
file_not_writeable_ok "path/to/index.html"; 
my $backup_file = "/usr/local/backups/myapp-backup.tar.bz2";
file_min_size_ok $backup_file, 911377;
 
# -M is a file test operator returning the file's last modifed age in days.
# See http://perldoc.perl.org/functions/-X.html
ok -M $backup_file < 1, "Backup file is newer than 1 day old";

Just run that script as a CGI or with NRPE etc. and you’ve got instant monitoring of things like backup success, which is otherwise kind of hard to hook in to Nagios. You could get even fancier with checksums or whatever you want.

See Test::Tutorial for more. Or look for TAP tools for your favorite language; they’re probably out there.

Tolerating ambiguity

In some cases, a certain amount of failure is acceptable. I’ve been monitoring connectivity between crucial hosts with a test script. I don’t want to be bothered by this check if one of our Citrix VMs isn’t available to the gateway server for a while, but it’s a problem if they’re ALL inaccessible because of a firewall issue or something. The configurable warning/critical thresholds on check_tap.pl allow me to define how much aggregate failure is worth getting excited about–something that’s hard to do with vanilla Nagios configuration.

Here’s the gist of my test script. ping_ok() and service_ok() come from a module I wrote 50 years ago to check basic connectivity/responsiveness with Net::Ping.

diag "Checking Citrix farm connectivity from DMZ\n";
my @stas = qw(
   citrixdc01.mydomain.com
   citrixdc02.mydomain.com
);
foreach (@stas) {
    ping_ok($_, 'http');	
    service_ok($_, 'http', "XML service on $_ is responding in some way" );
}
 
 
 
my @citrix_farm = qw(
   citrixps01.mydomain.com
   citrixps02.mydomain.com
   ...
   citrixps29.mydomain.com
   citrixps30.mydomain.com
);
foreach (@stas, @citrix_farm) {
    service_ok($_, '1494', "ICA service on $_ is responding in some way" );
}

I can just use the “warning” and “critical” arguments to check_tap.pl to set my thresholds for how many inaccessible servers I want to tolerate.

So there you go. Sysadmins and developers rejoice!

I’ve got Gnu Emacs 23.3 for Windows installed in c:/emacs, and a fairly recent install of Cygwin on Windows 7.

Saved this script in ~/cygemacs.sh

#!/usr/bin/bash
c:/emacs/bin/emacsclientw.exe -n -a "c:/emacs/bin/runemacs.exe" `cygpath -wa $@`

Then in my ~/.bashrc:

alias ec="~/cygemacs.sh"

Now I can be all like,

$ ec ~/.minttyrc

And it opens a new frame in my running Emacs (I have (server-start) in my ~/.emacs), or starts Emacs and opens the file if Emacs isn’t running yet.

See also: EmasClient at EmacsWiki.

Bonus tips

Yes I’m still using Subversion but also gitting going with Git.

And courtesy of The Lumber Room, in ~/.bashrc and others:

    export SVN_EDITOR='c:/emacs/bin/emacsclientw.exe -a c:/emacs/bin/runemacs.exe '

And silence that annoying “kill client buffer z0mgbbq?!?” warning:

  (remove-hook 'kill-buffer-query-functions 'server-kill-buffer-query-function)

I’m updating an old Symfony project to use Symfony 1.3/1.4 and Propel 1.5 through François Zaninotto’s sfPropel15Plugin.

Problem:

I followed the README, right? But…

$ ./symfony propel:build --forms
>> schema    converting "C:/web/myproject/config/schema.yml" to XML
>> schema    putting C:/web/myproject/config/generated-schema.xml
>> propel    Running "om" phing task
>> file-     C:/web/myproject/config/generated-schema.xml
>> autoload  Resetting application autoloaders
>> autoload  Resetting CLI autoloader
>> propel    generating form classes
PHP Warning:  call_user_func() expects parameter 1 to be a valid callback, class 'FooPeer' does not have a method 'getUniqueColumnNames' in C:\web\myproject\plugins\sfPropel15Plugin\lib\generator\sfPropelFormGenerator.class.php on line 485
PHP Stack trace:
PHP   1. {main}() C:\web\myproject\symfony:0
PHP   2. include() C:\web\myproject\symfony:14
PHP   3. sfSymfonyCommandApplication->run() C:\symfony1.3\lib\command\cli.php:20
PHP   4. sfTask->runFromCLI() C:\symfony1.3\lib\command\sfSymfonyCommandApplication.class.php:76
PHP   5. sfBaseTask->doRun() C:\symfony1.3\lib\task\sfTask.class.php:97
PHP   6. sfPropelBuildTask->execute() C:\symfony1.3\lib\task\sfBaseTask.class.php:68
PHP   7. sfTask->run() C:\web\myproject\plugins\sfPropel15Plugin\lib\task\sfPropelBuildTask.class.php:135
PHP   8. sfBaseTask->doRun() C:\symfony1.3\lib\task\sfTask.class.php:173
PHP   9. sfPropelBuildFormsTask->execute() C:\symfony1.3\lib\task\sfBaseTask.class.php:68
PHP  10. sfGeneratorManager->generate() C:\web\myproject\plugins\sfPropel15Plugin\lib\task\sfPropelBuildFormsTask.class.php:72
PHP  11. sfPropelFormGenerator->generate() C:\symfony1.3\lib\generator\sfGeneratorManager.class.php:126
PHP  12. sfGenerator->evalTemplate() C:\web\myproject\plugins\sfPropel15Plugin\lib\generator\sfPropelFormGenerator.class.php:106
PHP  13. require() C:\symfony1.3\lib\generator\sfGenerator.class.php:84
PHP  14. sfPropelFormGenerator->getUniqueColumnNames() C:\web\myproject\plugins\sfPropel15Plugin\data\generator\sfPropelForm\default\template\sfPropelFormGeneratedTemplate.php:34
PHP  15. call_user_func() C:\web\myproject\plugins\sfPropel15Plugin\lib\generator\sfPropelFormGenerator.class.php:485
... etc ...

Solution

I removed too much of the default config in propel.ini..

Make sure this original line is still intact in propel.ini, even though you’ve removed/commented out all the propel.behavior lines that point to the old sfPropelPlugin:

propel.behavior.default                        = symfony,symfony_i18n