It sucks when server disks fill up. I have Nagios set up to notify me well before it’s a problem, but it happens annoyingly frequently, and it’s always the same cause: log files.
Symfony’s production environments don’t create logs, but the development environments do, and for one project I had several batch jobs logging to the myproject/log directory. (Troubleshooting batch jobs without logs is just crazy).
I was going to use symfony log:rotate, but
- I would need one cron job for each log
- I had permission issues since some of the logs belong to apache and others to root
- Everything else on the machine, pretty much, is managed with the standard Linux utility logrotate. Logrotate has been around for about 174 Linux Years, and is common to most distributions.
So here’s how I did it using with logrotate.
So I have to get a certification to progress in my job, and it needs to involve Microsoft .NET. OK. So I’m working through the MCPD (Microsoft Certified Professional Developer) .NET web development training kit, available at Amazon, Barnes and Noble, etc. In theory it’s all I need prepare for the 3 exams for the MCPD, 70-536, 70-528, 70-547.
Thoughts so far:
- C# is surprisingly well-designed. Especially compared to say, PHP, which was only designed after the fact.
- I LOVE LOVE LOVE the elegant high tech tool metaphor gracing the cover.
- I love books instead of training, especially “online training”. But I need to map out a schedule and stick to it or it will get lost amid many other things.
So here, I went through the books and with Perl’s help, added up the estimates for each section, adding a 10 minute review on each chapter.
I manage a growing number of symfony-based Intranet apps that share a common look-and-feel. I use the same stylesheet and images on them all, and it I am nervous every time I make Yet Another Copy of the same files, knowing that I’m duplicating them and causing trouble for myself.
Also, as I wrote in my post about custom error pages, I have error pages set up for each, and I hate copying them. So I have invested some time into building a plugin containing the visual assets and the customized error pages, so that it’s easier to keep them uniform across all the apps I work on, and so that it will be that much quicker to build new projects.
I don’t think anyone outside my company will want my plug-in, but these instructions will hopefully be useful for anyone who wants to do the same thing.
As security increases, convenience tends to decrease. But in this case we gain some of both.
CONCEPTS
Problem 1: Brute force SSH attackers
I recently made a Linux machine accessible by port 22 from the Internet, so I could remotely administer it with SSH. Within a day or two, there were hosts from all over the Internet trying to log in using random usernames (‘root’, ‘dave’, ‘mysql’, ‘neo’) and random passwords.
Problem 2: Weak system passwords
System passwords are becoming easier to guess by brute force every time a new processor comes out. It doesn’t matter how complicated your alphabet soup is, it’s just not long enough, or won’t be long enough eventually. The bad guys also have botnets full of machines and all the time in the world.
Working around problems 1 and 2: Key authentication
An SSH key is complex (random) and much, much longer than your password (see Wikipedia for more details about how public-key cryptography works). You store the private part of your key securely on your client machine, and tell the server to trust only the public key that goes with it. BUT, you should really use a nice, long, complex passphrase to lock up your private key so that someone who gains access to it can’t get free access to all your servers, which leads to
Problem 3: I’m too lazy to unlock my inconvenient SSH key all the time
I log in to different machines all the time, from a bunch of different workstations. But, I have a long passphrase on my SSH key. Too long to type every time, especially if I’m just looking at something or transferring a file or two.
Solution to problem 3: A key agent
An SSH key agent lets you unlock your key and load it into memory once, and then automatically uses it to get you in to all your servers. SUPER convenient.
Problem 4: It’s confusing
I tried to explain it to a friend recently and found nowhere on the internets that explained it all in one place.
Coming up: detailed instructions for getting started with SSH keys and using a key agent on Mac OS X (command line), Windows (PuTTY and Cygwin commandline ssh), and Linux/Unix (command line).
I’ve always had a weakness for “preferences”. I spent hours going through all the available preference panes on the first Mac I had access to (an SE II I think, ca. 1989), tweaking this and that to get it to look as “customized” as possible. Hours setting up font and styles on school assignments, days banging on my Emacs config to be more like I want.
So it’s been secretly irking me for a while that although I installed the very nice WP-Syntax code highlighting plugin for this WordPress blog, it was hard to get it to display things in reverse-video like I prefer for all my terminal and text editor windows, and like I love in the online Symfony documentation. It’s just nicer on the eyes, you know? Digging in, I found the GeSHI syntax highlighter to be impressive in scope but scary in its inline-style-generating detail.
So today, to solve the problem for more than just me, I published my very first WordPress plugin, wp-syntax-hacktify. Very educational! I like the plugin system at wordpress.org.
The plugin tells GeSHI to use stylesheets instead and provides a slightly more commented/documented stylesheet as an example for if you’d like to override it with your own color scheme.
So now my code samples can be all nicer and stuff, like this:
/** * Print a string to the log using 'debug' level. For printf-style * debugging. * * @param string $m The string to log * @return nothing */ public static function debug ($m) { if (sfConfig::has('sf_logging_enabled') && sfConfig::get('sf_logging_enabled')) { if ($logger = sfContext::getInstance()->getLogger()) { $logger->debug($m); } } elseif (sfConfig::has('bhLDAP_echo_debugging') && sfConfig::get('bhLDAP_echo_debugging')) { echo "# $m\n"; } else { // echo $m; } } /** * Dump a data structure to the log at the 'debug' level. Uses * print_r() formatting. * * @param mixed $v The variable/data structure to dump * @param string $label An optional label to print in front of the dump * @return nothing */ public static function debugDump ($v, $label = "var dump") { self::debug("$label: " . print_r($v, true)); } |
Now the code on my blog can look nice even if it’s terrifically hacky!
For every Symfony project I work on, I eventually want to get rid of the tasteful brown error pages with the Symfony logo on them and install my own, so the users don’t get weirded out and somehow start thinking “Symfony = error”. Also, I’m probably overly optimistic, but it would be nice if there was a valid email address on the error page so I could hear users complain.
The documentation is good, but I want a simpler recipe that also accounts for all the forseeable errors, even those that don’t get to Symfony. Some details vary by version, but in short, for symfony 1.1: (more…)
Background
We need a quick-and-dirty
Simplifications
Unlike other similar implementations out there, I have smaller goals:
- The database will be updated/written by the HL7 interface and web interface. The LDAP tools don’t need to write to it.
- All my user data should fit nicely into two tables (users and groups… and another table to relate many-to-many), and two Objectclasses in LDAPland.
I found this information in this forum topic, “Slash in parameter” which references this ticket, but had snags and wanted to write it all up cleanly here.
The Problem
Sometimes a parameter in a GET string contains slashes, which are interpreted as file path delimeters by Apache and/or parameter name/value separators by Symfony.
So this URL works: http://myapp/employee/bytitle/title/System+Engineer
But this one returns a 404 (Page Not Found) error: http://myapp/employee/bytitle/title/Programmer/analyst
Wrong solution
Double-escaping the slash into ‘%252F’ (e.g. urlencode(str_replace('/', "%2f", $this->getTitle()))) works if the URL isn’t rewritten by Apache.
This threw me because my URLs would work if they included the name of the controller file, frontend_dev.php or backend.php or even index.php like http://myapp/index.php/employee/bytitle/title/Programmer%252Fanalyst,
but they wouldn’t work through the implicit, controllerless URLs like http://myapp/employee/bytitle/title/Programmer%252Fanalyst
Right solution
Both
- Turn on the AllowEncodedSlashes Apache directive by adding this line to your server-wide section or the appropriate VirtualHost:
AllowEncodedSlashes On
AND, - DON’T double-escape the parameter as above, but do use urlencode() or equivalent to change slashes to
%2F, eg. in the view layer,<?php echo link_to( $employee->getTitle(), '/employee/bytitle?title='. urlencode($employee->getTitle()) ); ?>
Additional Complication
Browsers may treat the encoded slash in the URL differently.
IE 7, 8 and Firefox 3, and Safari 3.2.1 on Windows display the URL in the address bar with the %2F intact, as you’d expect.
In Google Chrome 1.0.154.43, the links work but the address bar displays a slash instead of the encoded %2F, so if you hit the “Go” button you get a 404 error. Weird. Bug submitted!
Grrr, this cost me about 10 hours, and was hard to research, so I’m documenting it here for The Good Of Everyone:
Problem
I work with PHP on a bunch of different web servers. In this case I needed to resize photos (JPEGs) from Microsoft SQL Server (MSSQL) blobs on a Windows web server running IIS. I got stuff working on my local Windows development machine (under Apache, XAMPP rocks BTW), but when I moved to the production server:
- the script would silently fail with a 500 error whenever I called a GD function, specifically
imagecreatefromstring() - The most common IIS and GD error appears to be when PHP can’t find the GD DLL at all, which produces a nice message like
Fatal error: Call to undefined function: imagecreatefromstring() in c:\inetpub\wwwroot\test2.php on line 7 PHP Warning: Unable to load dynamic library ‘c:\php\ext\php_gd2.dll’ – The specified procedure could not be found. in Unknown on line 0
This was NOT my problem. My script kept failing silently, with no output after I called the first GD function. - IIS has no real error log like Apache. When my script silently failed, nothing was logged in the Application Event log, PHP’s error.log or the pathetic c:\WINDOWS\system32\LogFiles\HTTPERR\httperr*.log. Googling hard, I found I might have luck looking for “Worker Process Recycling Events”, which is apparently Microsoft’s IIS term for that confusing word “error”, but I didn’t go far down that path.
- Running my script from the command line using PHP didn’t produce any visible error either
IIS sucks. Apache’s error log is way better.
Solutions
I had to do both these things:
- Copy the php_gd2.dll to C:\WINDOWS\system32. Ridiculous, but I have had to do it before with other PHP modules to have them really work. After I did that, the script would work for small JPEG images.
- Increase the memory_limit in php.ini from 16MB to 32MB, so any whole high-res JPEG from the database will fit uncompressed in PHP’s memory
