Custom Filename Completion in Term::ReadLine::Perl

When we released the last version of our product, I got an IM from a support guy saying that we had broken filename completion in the installer. My reaction was “but we don’t implement filename completion!” I ran the installer on the previous release, and sure enough, it supported filename completion. Wha?

Turns out we were getting filename completion as a side effect of using Term::ReadLine::Perl to do prompting. Support “knew” we supported it, and even had a list of known bugs they passed around, but no one in dev knew about the feature or the bug. Good times!

Anyway, fixing what we broke on completions, I started looking at the bugs support had been informally floating around. Essentially there two issues, which caused three separate problems:

1) After every completion, Term::ReadLine::Perl inserted a space character after the path
Problem 1: When auto-completing paths, this meant you had to hit delete after each path completion before you could type the next few characters.
Problem 2: If you didn’t manually delete the space from the end of the completed filename, our installer would treat the space as part of the filename and tell you the file didn’t exist.

2) By default, after every completion, Term::ReadLine::Perl “decorates” the completed file
Problem 3: If the file you were completing to was a symlink, it would have ‘@’ appended to it, and our installer would tell you the file didn’t exist.

Problems 2 & 3 were fixable with some logic after the user hit enter, and problem 1 wasn’t that annoying, but I went looking to see if I could fix the actual completion function in Term::ReadLine::Perl. I did figure it out, but since there seems to be little google-able documentation specific to addressing this in Term::ReadLine::Perl (as opposed to Term::ReadLine::Gnu) I thought I’d document what I did.

Here’s my toy app demonstrating the above issues:

1
2
3
4
5
6
$| = 1;
use Term::ReadLine;
my $Term = new Term::ReadLine 'Installer';
 
my $file = $Term->readline('Enter a file path: ');
print "$file = '$file'n";

The relevant library code is in Term/readline.pm. This appears to be a Perl5 wrapper around Perl4 code.

The default filename completion function is called rl_filename_list(). This function has a tunable to turn off the filename decoration (adding “@” to symlinks, etc) called $var_CompleteAddsuffix, set to “1” by default. I tried turning it off like this at the top of my app:

$readline::var_CompleteAddsuffix = 0;

This worked, in that it turned off decoration. Unfortunately, the adding of “/” to a completed path is part of decoration, and that’s functionality I wanted to keep. So, I guess I need to write my own filename expander. Here is my toy app with my custom filename expander routine:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$| = 1;
use Term::ReadLine;
my $Term = new Term::ReadLine 'Installer';
 
# tell Term::ReadLine::Perl to use my filename expander for completion
$readline::rl_completion_function = "main::rl_filename_list_lcl";
 
my $file = $Term->readline('Enter a file path: ');
print "$file = '$file'n";
 
exit;
 
sub rl_filename_list_lcl {
    my $pattern = $_[0];
    my @files = (<$pattern*>);
    foreach (@files) {
        if (-d $_) {
            $_ .= '/';
        }
    }
    return @files;
}

The subroutine rl_filename_list_lcl() is a slightly modified copy of readline::rl_filename_list(). I removed all the decoration except ‘/’ for dirs, and I removed the check of $var_CompleteAddsuffix before decorating (I want it to happen unconditionally).

That solved the decoration issue, but not the “adds a space after every completion” issue. This is controlled by $rl_completer_terminator_character being unconditionally set to ‘ ‘ in readline.pm’s complete_internal(). I experimented with setting it to '' unconditionally, which did fix my problem. However, I didn’t want to leave it like that unconditionally for fear of unexpected side effects. Then I came across this in the code:

##   $rl_completer_terminator_character -- what to insert to separate
##      a completed token from the rest.  Reset at beginning of
##      completion to ' ' so completion function can change it.

Perfect! I’m already defining a custom function, so all I have to do is set $readline::rl_completer_terminator_character to '' in it.

Et voila! My completed toy function with auto-complete behaving the way I want it to:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
$| = 1;
use Term::ReadLine;
my $Term = new Term::ReadLine 'Installer';
 
# tell Term::ReadLine::Perl to use my filename expander for completion
$readline::rl_completion_function = "main::rl_filename_list_lcl";
 
my $file = $Term->readline('Enter a file path: ');
print "$file = '$file'n";
 
exit;
 
sub rl_filename_list_lcl {
    my $pattern = $_[0];
    my @files = (<$pattern*>);
    foreach (@files) {
        if (-d $_) {
            $_ .= '/';
        }
    }
 
    # The readline.pm library sets this to ' ' on every completion call, we can
    # force it to '' here but have to do it every time
    $readline::rl_completer_terminator_character = '';
 
    return @files;
}

Alternating Test Step Row Colors in TestLink

Working on implementing TestLink at work for better testing process flow. Very few complaints about the product as a whole – it can feel ponderous at times, but testing is a big problem, and TestLink is the best tool I’ve found for addressing them all.

The only real negative feedback I’ve received so far is regarding the display of steps from individual test cases. By default, the entire table is the same color, and it’s very difficult to visually track which row is which. It seems like such an obvious problem that I assumed I had missed a knob to tweak, but I can’t find anything about it in the docs, the forums, or the bug tracker.

I poked around a bit tonight and the solution turned out to be ridiculously (but pleasingly) simple.

Update 2012-05-08: There is a usable-with-the-patch-command patch available that incorporates all of these changes and more. See the README at the base of the github repo for more details on what’s in that patch.

First, add the following to gui/themes/default/css/custom.css:

gui/themes/default/css/custom.css
1
2
3
4
5
6
.row_color_1 {
    background-color:   #DDDDDD;
}
.row_color_2 {
    background-color:   #CCCCCC;
}

Then, patch the following two files in gui/templates/testcases/ like so (original files were from 1.9.3):

inc_steps.tpl
1
2
3
4
5
6
7
8
9
10
11
--- inc_steps.tpl.dist  2011-08-22 21:08:21.000000000 -0500
+++ inc_steps.tpl       2011-08-22 21:13:02.000000000 -0500
@@ -45,7 +45,7 @@
        </tr>
        {* BUGID 3376 *}
        {foreach from=$steps item=step_info}
-       <tr id="step_row_{$step_info.step_number}">
+       <tr id="step_row_{$step_info.step_number}" class="{cycle values="row_color_1,row_color_2"}">
                <td style="text-align:left;">
                        <span class="order_info" style='display:none'>
                        <input type="text" name="step_set[{$step_info.id}]" id="step_set_{$step_info.id}"
tcStepEdit.tpl
1
2
3
4
5
6
7
8
9
10
11
--- tcStepEdit.tpl.dist 2011-09-22 10:27:23.000000000 -0500
+++ tcStepEdit.tpl      2011-09-22 10:39:55.000000000 -0500
@@ -197,7 +197,7 @@
   {* this means we have steps to display *}
   {if $gui->tcaseSteps != ''}
        {foreach from=$gui->tcaseSteps item=step_info}
-         <tr id="step_row_{$step_info.step_number}">
+         <tr id="step_row_{$step_info.step_number}" class="{cycle values="row_color_1,row_color_2"}">
       {if $step_info.step_number == $gui->step_number}
                    <td style="text-align:left;">{$gui->step_number}</td>
                  <td>{$steps}</td>

Blow away the cached .css files, and voila! Alternating background colors for test steps in TestLink.

Before: After:

I tried to post this as a hint in the TestLink forum but I honestly couldn’t figure out how to register a forum account. I did post a comment on a related bug in their issue tracker, ticket 3583

EDIT 2011-09-22: The original version of this post only included a change to inc_steps.tpl. Added the patch to tcStepsEdit.tpl to apply alternation to the rows in the test case step edit view also.

EDIT 2011-09-26: Updated to include the correct full path to custom.css

Horizontal scrolling in code blocks

Before I forget, to get the nice horizontally-scolling code block in my previous post, I had to added the following to my wordpress theme’s style.css:

1
2
3
4
5
6
.hentry code {
  display: block;
  background: lightgrey;
  padding: 10px;
  overflow: auto;
}

Then I can get the nice boxes by surrounding code snippets with both pre and code blocks. pre gets me monospaced font, a theme-consistent font size, etc. code gets me the scrolling and the grey background.

Oh, I did also comment out the “letter-spacing: -1px;” from my theme’s pre definition, it made the code look squished.

Got pointed in the right direction by these two sites.

Since I will not be able to remember in the future: wp-content/themes/clean-home/style.css

Enforcing a default owner for new tickets in Trac

We’ve been loving our Trac install at work, but we are still sanding down a few rough spots.  One particular spot that’s been bobbing up has been what happens when “new” tickets have an owner assigned.  I liked the ticket status being “new”, because it meant that I could easily look at my ticket list and see tickets that had been directly assigned to me that I hadn’t acknowledged yet by accepting them.  Someone else really disliked having to “accept” a ticket that was already assigned to him before he could move it to QA.

I actually managed to create a solution for this using the TicketConditionalCreationStatusPlugin from TracHacks.  This plugin allowed me to conditionally set the status of the ticket based on the contents of the “owner” field.  This was actually a pretty slick solution, but for various reasons it didn’t sit well with a lot of people.

The next idea we had was to prevent new tickets from being created with any owner besides the default value of “dev” (our dev team mailing list).  This solution had three main benefits – It worked the same for everyone; no one would ever have a “new” ticket assigned to them; and the “dev” alias was always associated with a new ticket, which has some value in relation to email notifications.

Easy enough, right?  Maybe it’s just me, but I could not find a good way to enforce this.  There’s no native ability to hide fields that aren’t drop-downs, and owner’s not a drop down by default.  It can be made a drop-down, but not in a manner which would allow it to be removed.  There are several owner-oriented plugins, but none of them differentiate between the “new ticket” screen and the “modify ticket” screen.

After a bit of fruitless experimentation I realized that I was overthinking it.  I made the following change:

templates/site.html
- <body py:match="body" py:attrs="select('@*')">
+ <body onload='document.getElementById("field-owner").disabled = true;' py:match="body" py:attrs="select('@*')">

et voila!  My default owner value is visible but immutable!

I know this isn’t a rock-solid solution, but in our environment (restricted corporate access) I’m not worried about evil-doers and griefers, I’m just wanting to remind people that they’re not supposed to change the value…

Hardware is hard, let’s go shopping…

After a brief diversion into soldering I was back to playing with the Arduino last night, working through samples in the Arduino Experimentation Kit I bought a while back from Adafruit.  I’m only up to experiment 4 because I’ve spent a lot of time trying to understand the underlying electronics theory (why can’t you drive a motor directly from an arduino pin?  I mean, mathematically?  What the heck does a transistor actually do?).  The experimentation kit seems geared more towards getting people comfortable with the programming than the electronics, so wikipedia and I have been spending a lot of time together.

Last night’s experiment was working with a shift register to drive 8 LEDs.  There was a previous 8 LED project, but those were driven directly from Arduino pins, this project was designed to drive the same 8 LEDs, but only using 3 pins.  Wiring it was a snap (more or less, still trying to get a grip on the best order in which to place components) and the software was easy to load.  Just needed to give it a cursory run to prove it works so I could move on to what I really wanted to do, which was to see if I could figure out how to load a potentiometer into the circuit and use the LEDs as an indicator of the pot’s position.

Compile, load, and… Well, ok, the light’s were blinking, but not in the order I expected.  Changed the code to increase the delay between cycles, and the blinks were definitely not what I expected.  Rereading the project details, I notice it never actually says what the lights will do, but it seems really clear from the code that it will be a 0-255 binary counter.  But that’s not what it’s doing.  I spent an hour reviewing my code (which means I would have found a bug in the kit’s sample code, which seemed unlikely), removing the loop, trying to drive just one led for the whole app…  Nothing.  The pattern of the lights would change everytime I made a change, but not in any manner I could predict.

It was late so I decided to hang it up for the night and try again later.  I put everything away, and was standing up when I happened to glance at the wiring diagram.  I instantly noticed I had reversed the CLOCK and LATCH pins on the shift register.  The Arduino still had the last sketch I loaded on it (which simply sent “8” to the shift register), so I plugged in the 9v battery and voila – LED #3 lit up.  Arggg!

It seems likely that it will take me a while to get used to the rhythms of electronics/microcontroller debugging.  95% of the time I know where to look for a bug in a piece of software I’m writing instantly, but because the lights did something, I assumed the circuit was right and the code was wrong.  Bzzt.  That would be like a junior developer telling me his code wasn’t working, and he was pretty sure it was a bug in the library.  Nope.

On the plus side, now I’m ready to play with the pot tonight!  Whee!

When did the metaverse die?

NPR published a reader-selected list of the top 100 Science Fiction and Fantasy books.  The list is wonderful and great fun can be had finding new stuff to read or debating the relative merits of the books.

I was thrilled to see one of my all-time favorites on the list, Snow Crash by Neal Stephenson.  If you haven’t read this book and you care at all about science fiction at all, you’d probably enjoy it.

However, I was taken off-guard by their book description:

“Weaving contemporary imagery with Sumerian myths, Stephenson’s third novel revolves around a mysterious “pseudo-narcotic” Snow Crash that is capable of affecting people both within — and without — the alternate-reality Internet called the “Metaverse.””

The Metaverse of the book is essentially the same virtual reality that everyone was discussing.  I think you used goggles to get to it instead of “jacking in” ala Neuromancer, but same general concept.  I know I’m reading to much into a 40-word blurb, but doesn’t it seem to say that the quest for virtual reality is over because we have the internet?  Bah!

I still hope for compelling, readily available immersive computing environments.  Of course it will be over the Internet, or whatever replaces the Internet as a communications bus, but just as the world wide web isn’t defined as the Internet itself, so too will VR ride on top of and extend the Internet.

Please, please don’t say anything to imply that the Metaverse is already here.  I want my house!

Blog Necromancy…

I’ve been enjoying posting random things on Google+ but have been feeling vaguely guilty about clogging up the tubes.  Decided to start up a blog to post anything I wanted guilt free.  While doing so I remembered an old blogspot blog I had for a while.  Everything before this post was imported from that (since deleted) blog.

I have the left and right…

I was playing around with the Online Etymology Dictionary tonight and was randomly typing in short words. In the entry for “jack” I found this:

The jack in a pack of playing cards (1674) is in Ger. Bauer “peasant.”

Huh… now what am I going to idly wonder about as my grandparents hand me my ass in Euchre? Wonder if the connection I’m assuming is real. Better check Wikipedia:

Euchre, the game responsible for introducing the Jack (Joker) into modern packs, invented around 1860 to act as a top trump or best Bower (from the German word Bauer, “farmer”, denoting also the Jack at cards[1])

That sounds promising, but it’s possibly lore since it assigns a bit of a heightened sense of worth on the subject of the article for introducing a major element into modern decks. So let’s check Playing Cards:

Before this time, the lowest court card in an English deck was officially termed the Knave, but its abbreviation (“Kn”) was too similar to the King (“K”) and thus this term did not translate well to indices. However, from the 1600s on the Knave had often been termed the Jack, a term borrowed from the game All Fours where the Knave of trumps has this name. All Fours was considered a game of the lower classes, so the use of the term Jack at one time was considered vulgar. The use of indices, however, encouraged a formal change from Knave to Jack in English decks

.

Hmmm… That implies that the “role” of the Jack had been played by the Knave for a long time, and that it had been unofficially termed “Jack” since the 1600s. That’s not so promising. While the entire concept of what a deck of playing cards has always been fluid, this section seems to clearly show that the concept of three “court” ranks was already in place in the 1300s:

It is likely that the precursor of modern cards arrived in Europe from the Mamelukes of Egypt in the late 1300s, by which time they had already assumed a form very close to that in use today. In particular, the Mameluke deck contained 52 cards comprising four “suits”: polo sticks, coins, swords, and cups. Each suit contained ten “spot” cards (cards identified by the number of suit symbols or “pips” they show) and three “court” cards named malik (King), nā’ib malik (Viceroy or Deputy King), and thānī nā’ib (Second or Under-Deputy)

Well, it looks to me like Euchre introduced neither the concept of the Jack, nor the name “Jack” itself. The headscratcher for me is the reference for the original statement, which is “Oxford Dictionary of Card Games, David Parlett – pg.104”. I can’t find the text of that reference online, but the author seems like a serious scholar from what I turned up. It makes me wonder if there wasn’t a misunderstanding by the reader of what the writer meant.

Oh well, it was a nice story while it lasted…

Home Improvement

I just spent all day today elbow deep in water trying to fix a copper line I broke while trying to change out a toilet in the basement. This project started when I suggested that I should build some shelves in the basement for storage. It made sense at the time…