Generating PCRE Regular Expressions from Date Format Strings in PHP
Universal #2
guypaddock
This post has been moved to my RedBottle Blog.

Java-inspired Enumerated Types in PHP
Universal #2
guypaddock
This post has been moved to my RedBottle Blog.

Plesk Permissions: Hosting performance settings management
Universal #2
guypaddock
The client permissions system in Plesk Panel is a bit of a mess – it's not fine-grained enough, and many of the permissions are either only vaguely described in the documentation, or don't control what you want them to control.

One such permission is "Hosting performance settings management"; the help documentation describes it as "Specify whether the customer will be able to limit bandwidth usage and number of connections to his or her Web sites," but if you enable it on its own, you will find that the client still won't have the ability to set their instantaneous bandwidth and number of connections under the "Performance" page for a domain. As it turns out, the permission has no effect unless the client also has the "Physical hosting management" permission (which, as is typical for Plesk usability, is not near the bandwidth permission, but is at the very top of the page, despite being a related permission).

Requiring FTP users to be in the "ftp" group when using VSFTPD
Universal #2
guypaddock
I just installed VSFTPD on Red Bottle's Gentoo-based server, and wanted to restrict ftp access to just user accounts in the "ftp" group. Unfortunately, I found that the only way to allow or deny access to specific users via the VSFTPD config file (vsftpd.conf) is to provide a userlist_file directive, which would mean I'd have to list all the users I want to have access to FTP in a special file just for VSFTPD. Translation: one extra file to update every time we add a new user who needs FTP access, which isn't ideal.

Fortunately, VSFTPD supports PAM, which means that I was able to add a PAM requirement that the user be in the "ftp" group, allowing me to completely bypass adding users to a VSFTPD user list file. To do this, I added the following lines to /etc/pam.d/ftp:
# GAP on 2010-08-06: Require FTP users to be in the FTP group
account  required  pam_succeed_if.so user ingroup ftp


That's all there is to it! Once you add those lines, FTP access should be restricted to user accounts in the "ftp" group.

A fix for the "You must restart..." error in Windows Media Player DRM
Shark Attack
guypaddock
I use a modified Sony VAIO UX-390N Ultra-Mobile PC in the car, and I love it. I have a subscription to the Rhapsody music service, which provides me with all the music I could ever want (except for those particular tracks they can't ever seem to secure the licenses to...), and I use StreetDeck to make it easier to play the music in the car.

Unfortunately, the past year or so, I've run into an extremely frustrating problem with the DRM in Windows Media Player that would limit me to being able to play only about 30 or so tracks while inside StreetDeck, including tracks I skipped through! It seemed that after hitting the magic number of tracks (which might be 21, 25, 30, etc), I would get this message:

Windows Media Revocation and Renewal

You must restart the machine to continue content processing.


What made the problem even more frustrating was that, just as the message indicated, the only fix for it was to restart the machine. If I didn't, I couldn't play any more licensed tracks in either StreetDeck or Windows Media Player for the rest of the Windows session.

Curiously, playing the same tracks in the Rhapsody client (even without an internet connection at the time) did not trigger the same issue. I thought that this was a sleazy limitation that was recently added to Windows Media DRM, to prevent users from using any other software than what the content provider prefers for playback, and I've been actively trying to circumvent it for months.

Luckily, it appears that this little "feature" of WMDRM might actually be a legitimate bug. According to a MuvEnum FAQ entry about this issue, it seems that many different things can cause it to occur. As it turns out, in my case the theory is that WMDRM can get a false positive from things like the "Tablet PC Input Service" (TIP) in Vista and Windows 7, which can cause WMDRM to think that the user has some sort of automation program which he or she could be using to programmatically strip DRM off his or her music collection. Perhaps WMDRM keeps an internal count of how many tracks are played when an automation program is detected, and cuts the user off after a certain limit is reached.

From the FAQ entry, I've found two ways to solve the issue. I can do one of the following:

Option 1: Run StreetDeck and Windows Media Player as an Administrator


It seems that running the player as an administrator fixes the issue. Now, of course, Rhapsody and others blame UAC for the cause of the problem entirely, but my guess is actually that the bug has nothing to do with the player having insufficient privileges.

Instead, the reason why running Windows Media Player as an admin appears to fix the issue could be that it isolates the user-space Tablet PC Input application from the WMP application interface. For security reasons, a program running as an admin can't receive input from an application that's not running as an admin, and I think the TIP applications are subject to the same restriction. I don't think you can use Ink to interact with an admin program. So, it could be that when WMP is run as an admin, that's enough to keep it from being threatened by the Table PC input service application.

Option 2: Temporarily Disable Tablet PC Input Service During Playback


The other option is just to stop the "Table PC Input Service" service from within an elevated instance of Task Manager prior to starting playback. This way, the player is still secure because it doesn't have admin privileges.

I like this option more than option 1 because I don't want to have to approve a UAC prompt every time I start StreetDeck in the car, and I don't trust StreetDeck enough to let it run with full privileges for any length of time. For starters, the app depends on UAC Virtualization to prevent it from writing in my "Program Files" folder, and StreetDeck is buggy enough that I don't want it writing there.

I will see if I can come up with a batch script of some sort that can temporarily disable TIP while StreetDeck is running.

Dispatching cron tasks more frequently than every minute
Universal #2
guypaddock

Here at Red Bottle, we use a system called Aegir to manage our Drupal installations. One of its components is the "Task Queue", which keeps track of the operations that an Aegir user has queued-up through the user interface. In a normal Aegir install, you would use cron to schedule the task queue to execute once every minute or every five minutes, but since we do Drupal provisioning so often, one minute is an eternity!

Unfortunately, cron doesn't let you schedule a task any more frequently than once per minute. What if you want a task to run every fifteen seconds? Well, you could schedule the same command to execute four times each minute, like so:

*/1   *   *   *   *   my_command
*/1   *   *   *   *   sleep 15; my_command
*/1   *   *   *   *   sleep 30; my_command
*/1   *   *   *   *   sleep 45; my_command

This works for a while (it's actually the first solution we used), but it isn't very efficient because you can have up to four processes (the command and the three sleeps) running at any one time, and those processes use-up memory and file handles.

Instead, try a script that we wrote called dispatch_per_minute.php.

To use it, download it, save it to a file on your machine without the ".txt" extension (ours is /usr/local/bin/dispatch_per_minute), and then make it executable (chmod a+x dispatch_per_minute). To test it out with debugging on, try this:

dispatch_per_minute 4 "echo Hello World\!" --debug

You should see the following:

Sleep time will be 15 seconds.

Executing the command: echo Hello World\!
Hello World!
Sleeping 15 seconds...

Executing the command: echo Hello World\!
Hello World!
Sleeping 15 seconds...

Executing the command: echo Hello World\!
Hello World!
Sleeping 15 seconds...

Executing the command: echo Hello World\!
Hello World!
Total time: 00:45

When you're ready to use it with cron, add a line like the following to your crontab:

*/1   *   *   *   *   /PATH/TO/dispatch_per_minute 4 "my_command"

The 8-hour Drupal / CCK multiple-value problem
Universal #2
guypaddock
I just spent an entire workday trying to track down a frustrating problem with a Drupal site we're creating, and the solution is so mind-bogglingly simple that I figured I'd share it with the world, in the hope of sparing others the same fate.

The Problem


I have a custom CCK multiple-value field that I wrote to keep track of line items in an invoice. I wrote the field module about two weeks ago, thoroughly tested it at the time, and thought that it was bug-free and good to go. Unfortunately, this morning I discovered that wasn't the case when I tested it as part of a larger system.

The issue started out when I tried to add a seventh line item to the field with the "Add another item" button that CCK automatically provides for unlimited length, multiple-value fields. Instead of the normal Ajax / AHAH callback followed by a new line item, instead I got an error message with HTTP code 500. "Oh no!" I thought -- something's wrong with the site or the code, and off to the error log I went.

The error in the log was actually from MySQL, and it was complaining "Got a packet bigger than ‘max_allowed_packet’ bytes" for a query to update the form cache ("cache_form" table). After reading a bit, I discovered that this can be the case for larger Drupal forms, and that the accepted solution was to increase MySQL's maximum packet size. I found that the server was configured for the default size of 1 MB, and I increased it to 16 MB. I was now able to add two more line items to my field before getting a 500 error again. Sadly, I realized that increasing the packet size still didn't fix the underlying issue.

Now, instead of MySQL complaining, PHP was complaining that it was running out of memory, even though the memory limit was already 256 MB (it wanted to allocate an additional 50 MB). Okay, something didn't feel right, but I increased the memory limit to 312 MB anyway and tried it again. Still, same issue, but now PHP wanted about 64 additional MB. Give it an inch and it wants a mile. Okay, fine, so I tried 1 GB as the memory limit, and now PHP wanted to allocate an additional 150 MB. Ugh. Now I was wondering if I had a memory leak or some other obscure problem with CCK that you only ever find one or two other people have ever encountered and that the maintainers think is impossible.

With a little more investigation, I found out that the memory limit was being hit when PHP was trying and failing to print out an error message containing the failed SQL query to update the form cache. The query was incredibly long, and it was being passed through check_plain(), which was causing PHP to allocate space to hold two copies of it in memory. Apparently, that required a LOT of memory.

Why was the query failing? Well, the system this site runs on is a MediaTemple Virtual Private Shared (VPS) server running on Virtuozzo, and MediaTemple has several quotas in-place to ensure that each virtual server behaves itself and doesn't use too many resources. One such quota is called "othersockbuf", and it apparently controls the maximum size of a socket buffer between local processes. MediaTemple sets it at 1433738 bytes (roughly 1.37 MB), which meant that no matter how large I set the MySQL maximum packet size, the maximum length of a query would still be 1.37 MB. I caught on to this because I was watching "/proc/user_beancounters", and whenever I got the 500 error, the fail count for "othersockbuf" would increase. Basically, the query must have been 2+ MB, and Drupal was failing to transmit the whole thing before hitting the quota. Then, in the process of displaying an error message about the issue, it was running out of memory.

Okay, fine, so now I knew what was causing the behavior that I was seeing in the logs, but what was the underlying cause? What would cause the query to be so large?

I decided to poke around in the form cache table to find out what data was being stored. To do that, I refreshed a node edit form containing my custom field, grabbed the build ID, and wrote a custom little PHP script to query the form cache table, pull out the entry for the form that was just constructed, unserialize the data, and pretty-print it to a file. Next, I had Drupal add a new line item to the field, used the script to dump the data to a second file, added yet another new line item, and dumped the data to a third file.

The first thing I noticed was how much of a size discrepancy there was between the dumps. The first file was only about 125 KB, the second was 609 KB, and the third was 9.3 MB (!). So, adding one line item increased the size of the form state by a factor of 5, while adding a second increased it by a factor of 15! Clearly, there was duplicate data.

When I compared the files, I found that the "#default_value" of each subsequent line item was even deeper than the previous one, and that there were multiple copies of all of the values for the entire field in each one. Clearly, something was wrong with the way that values were being nested / un-wrapped.

The Solution


As it turns out, the solution is incredibly simple, and I feel stupid for taking this long to diagnose it.

Essentially, the hook_widget() implementation for the widget that accompanies the field looked like this:
function my_widget(&$form, &$form_state, $field, $items, $delta = 0) {
  $element = array(
    '#type'             => 'my_widget_element_type',
    '#default_value'    => $items,
  );

  return $element;
}

This was modeled after similar logic that CCK itself employs in nodereference_widget() for the "nodereference_select" widget, where all of the items of the field are passed on to the custom FAPI element type (which is "my_widget_element_type" in this case).

Then, my implementation of hook_elements() looked like this:
function my_elements() {
  $elements = array(
    my_widget_element_type => array(
      ...
      '#process'  => array('my_element_process'),
      ...
    ),
  );

  return $elements;
}

And, finally, my process callback looked like this:
function nodereference_line_item_element_process($element, $edit, &$form_state, $form) {
  $delta = $element['#delta'];

  $element['inner_field'] = array(
    ...
    '#default_value'    => isset($element['#value'][$delta]) ? $element['#value'][$delta] : '',
    ...
  );

  return $element;
}

Do you see the problem? Don't feel bad, I didn't either, but there definitely is one.

As it turns out, the issue is that the hook_widget() implementation passes the entire $items array, instead of passing $items[$delta]. You see, hook_widget() gets called for every line item, not just the entire field, so by passing the entire $items array, the default value for each line item would end up being the values for the entire field. Thus, if the form contained 5 lines already, and I added another one, the form state now held 6 copies of each of the values of the 6 line items (NxN values, with N being the number of lines), which explained the dramatic increase in memory size.

The reason the control worked for a short time, though, was because the process callback was populating the default value of the inner control with just the value for the appropriate delta (i.e. the specific line item). CCK's own Node Reference module doesn't have to worry about passing the entire $items array because it handles its own multiple-values (each value becomes an item for selection in the drop-down box, so there's only one call to hook_widget() for the entire field).

The fixed hook_widget() implementation looked like this:
function my_widget(&$form, &$form_state, $field, $items, $delta = 0) {
  $element = array(
    '#type'             => 'my_widget_element_type',
    '#default_value'    => $items[$delta],
  );

  return $element;
}

And the fixed process callback looked like this:
function nodereference_line_item_element_process($element, $edit, &$form_state, $form) {
  $element['inner_field'] = array(
    ...
    '#default_value'    => isset($element['#value']) ? $element['#value'] : '',
    ...
  );

  return $element;
}

I hope this helps someone else, because it sure was a fucking picnic to track down.

Quick Drupal Tip: Use form_error() instead of form_set_error()
Universal #2
guypaddock
Here's a quick tip I just learned today: when doing custom form validation (i.e. in an #element_validate callback, or in a _validate() form function), use form_error() to easily flag an error on form element, rather than using form_set_error(). form_error() automatically handles imploding the #parents and then calls form_set_error() for you.

This can save you about 1 line of code per validation error, or shorten up your existing long line, if you're one of those developers who chains calls all on the same line.

Customizing the Forgotten Password / log-in error messages with WordPress
Universal #2
guypaddock
This is just a quick note for people who have the unfortunate displeasure of using WordPress. If you have a client who needs you to change the error messages WordPress displays on the log-in and forgotten password reset pages, all you need to do is create a new plug-in that adds a hook for the login_errors filter.

The only argument passed to the filter function is the HTML text of the error message, which you'll probably want to pass through strip_tags() and trim() before using in comparisons (to make your life easier).

After stripping and trimming, the text of the different error messages are:
  • ERROR: The password field is empty.
  • ERROR: Invalid username. Lost your password?
  • ERROR: Incorrect password. Lost your password?
  • ERROR: Invalid username or e-mail.
  • ERROR: There is no user registered with that email address.
Your filter function must return the HTML error text to display for the error, even if you haven't modified the error message you were passed.

Funny Duraflame Ad
Universal #2
guypaddock
I saw this on the WeatherBug website, and thought it was a pretty creative way to sell fire logs (click on it, then wait for it to load):

?

Log in