It’s bugged me for a while that my WordPress posts would often have an extra space at the beginning of a line. It would only happen when it was a new sentence, and after a while I realized it was related to my habit of typing two spaces after a period.

Fear my drawing skills...

Continue reading →

Yesterday I worked on streamlining our ticket workflow in Trac. While I was in the test project anyway, I decided to address an issue that had bugged me since I first set up Trac. I originally set up two custom ticket fields related to case IDs in our CRM system:

crmnum = text
crmnum.label = Case #
crmnum.format = plain
crmnum.order = 1
crmlink = text
crmlink.label = Case Link
crmlink.format = wiki
crmlink.order = 3

The idea was that we could have the “crmnum” field to just hold the ID, which we could act on programmatically. The “crmlink” field would be a copy/paste link of the ticket’s URL, which would allow someone looking at the ticket to click through to the CRM case easily.

The problem with this is that the link is long and ugly and completely throws off the tidiness of the ticket summary.

So, having recently figured out how to write a custom wiki macro, I decided to create one for links to our CRM system also. The idea would be that the “crmlink” field would go away and the “crmnum” field could be updated to use the wiki Macro to generate the link automatically.

I nuked the “crmlink” field and changed “crmnum” from “type = plain” to “type = wiki“. I wrote my custom CRM macro, testing it in the wiki itself until I had it behaving. I then created a new ticket, entered [[CRM(123)]] in the crmnum field, and clicked “Preview”. Instead of seeing the link it should have returned, instead I got [[CRM(...)]]. I did some testing and proved to myself that some wiki processing was taking place (ticket:1 expanded correctly) and that no macro expanded in that field, including built-in macros.

Then on a hunch, I changed crmnum to “type = textarea” instead of “type = text“. Sure enough, macros work properly in textarea fields but not text fields. In a way this makes sense since almost all built-in macros return multiple lines of text. The documentation for custom ticket fields doesn’t mention that wiki processing behavior differs between these field types. Unfortunately the textarea fields can’t easily be tricked into looking like a simple text field. They come with an editing bar, and they are assigned the same class as the description field, so you can’t easily use CSS tricks on it.

I was unsure if this was a bug or intentional, but I managed to find the original Trac ticket for adding wiki processing to custom fields, and it’s clear that it was intentional to make the wiki-processing behavior of the two field types differ. Specifically, text fields are processed with format_to_oneliner() and textareas are processed with format_to_html() (this can be seen in the change to web_ui.py associated with new feature.

After figuring out that the text/textarea parsing differences were done intentionally, I lost interest in patching the code itself. Instead I looked into using InterWiki links instead of macros, which are processed in text fields.

I created trac ticket 10352 about this issue, hopefully it gets addressed one way or another.

Last week I posted about a custom [[TestLink()]] macro I had written for Trac. Yesterday I wrote another macro to do something similar, but in doing so I found out that I was probably doing a lot more work than I needed to.

The macro I wrote allowed me to write [[TestLink(m3-123)]] to link directly to test case 123 in the m3 project. It was about seven lines of python and not the most complex thing in the world, but nowhere near as simple as using an InterWiki prefix. I did this by adding the following line to my Trac install’s InterMapTxt wiki page:

TestCase https://devtest.example.com/linkto.php?tprojectPrefix=$1&item=testcase&id=$1-$2

This allows me to replace [[TestLink(m3-123)]] with testcase:m3:123.

There are tradeoffs of course. The macro allows full control using python. That allowed me to write a single macro that could produce links to different places in our CRM system based on the format of the single argument to the macro. Since I don’t get that flexibility with InterWiki links, I had to use two InterWiki links instead of a single macro.

On the other hand, the InterWiki links are simpler to write, look better, and are much closer to the standard Trac wiki link style that my users are already used to. Also, they have the advantage that they are expanded in ticket custom text fields and macros aren’t.

In the end I’ll keep both the macros and the InterWiki links for these cases since they aren’t mutually exclusive, but I’ll definitely turn to InterWiki links before I turn to writing a custom macro in the future.

I’ve been spending a lot of time lately trying to get the workflow between Trac and TestLink to make sense. I’m far from having the perfect solution, but a great first step was creating a Trac wiki macro that will link to specific test cases in TestLink. There’s lots of areas in which this could be improved in the future…

Create this as conf/TestLinkMacro.py in your project (replacing test.example.com with your TestLink host):

import re

from trac.wiki.macros import WikiMacroBase

class TestLinkMacro(WikiMacroBase):
    """Macro to generate automatic links to test cases in TestLink.
    """

    def expand_macro(self, formatter, name, args):
        """Expand a TestLink test case ID into a link
        """
        TestLinkServer = 'test.example.com'
        matches = re.match(r'(.*)-(d+)$', args)
        
        return '<a href="https://' + TestLinkServer + '/linkto.php?tprojectPrefix=' + matches.group(1) + '&item=testcase&id=' + matches.group(0) + '">[[TestLink(' + matches.group(0) + ')]]</a>'

After putting this in place, and possibly restarting Trac, you can create a direct link to test case m3-123 in a wiki by typing [[TestLink(m3-123)]]. Note that I could have changed the actual string displayed in the wiki to anything I wanted, but I chose to leave it looking identical to the actual macro (but linked) so that it would reinforce to people the correct format for the test case links.

This macro is useful in several ways, but the most obvious is allowing us to enter regression test case IDs into commit messages (which then get associated with a ticket in Trac and are displayed with the wiki engine, making the test case link clickable).
With commit messages being associated with tickets automatically and being displayed by the wiki engine, this gives us a nice syntax to enter regression case links into commit messages and be very followable later

Edit 2011-09-13:

See also this update to my use of macros for linking to test cases and this comment on custom ticket field wiki parsing, both of which have a direct relation to this post

When we released the last version of our product, I got an IM from a support guy saying that we had broken filename completion in the installer. My reaction was “but we don’t implement filename completion!” I ran the installer on the previous release, and sure enough, it supported filename completion. Wha?

Turns out we were getting filename completion as a side effect of using Term::ReadLine::Perl to do prompting. Support “knew” we supported it, and even had a list of known bugs they passed around, but no one in dev knew about the feature or the bug. Good times!

Anyway, fixing what we broke on completions, I started looking at the bugs support had been informally floating around. Essentially there two issues, which caused three separate problems:

1) After every completion, Term::ReadLine::Perl inserted a space character after the path
– Problem 1: When auto-completing paths, this meant you had to hit delete after each path completion before you could type the next few characters.
– Problem 2: If you didn’t manually delete the space from the end of the completed filename, our installer would treat the space as part of the filename and tell you the file didn’t exist.

2) By default, after every completion, Term::ReadLine::Perl “decorates” the completed file
– Problem 3: If the file you were completing to was a symlink, it would have ‘@’ appended to it, and our installer would tell you the file didn’t exist.

Problems 2 & 3 were fixable with some logic after the user hit enter, and problem 1 wasn’t that annoying, but I went looking to see if I could fix the actual completion function in Term::ReadLine::Perl. I did figure it out, but since there seems to be little google-able documentation specific to addressing this in Term::ReadLine::Perl (as opposed to Term::ReadLine::Gnu) I thought I’d document what I did.

Here’s my toy app demonstrating the above issues:

$| = 1;
use Term::ReadLine;
my $Term = new Term::ReadLine 'Installer';

my $file = $Term->readline('Enter a file path: ');
print "$file = '$file'n";

The relevant library code is in Term/readline.pm. This appears to be a Perl5 wrapper around Perl4 code.

The default filename completion function is called rl_filename_list(). This function has a tunable to turn off the filename decoration (adding “@” to symlinks, etc) called $var_CompleteAddsuffix, set to “1” by default. I tried turning it off like this at the top of my app:

$readline::var_CompleteAddsuffix = 0;

This worked, in that it turned off decoration. Unfortunately, the adding of “/” to a completed path is part of decoration, and that’s functionality I wanted to keep. So, I guess I need to write my own filename expander. Here is my toy app with my custom filename expander routine:

$| = 1;
use Term::ReadLine;
my $Term = new Term::ReadLine 'Installer';

# tell Term::ReadLine::Perl to use my filename expander for completion
$readline::rl_completion_function = "main::rl_filename_list_lcl";

my $file = $Term->readline('Enter a file path: ');
print "$file = '$file'n";

exit;

sub rl_filename_list_lcl {
    my $pattern = $_[0];
    my @files = (<$pattern*>);
    foreach (@files) {
        if (-d $_) {
            $_ .= '/';
        }
    }
    return @files;
}

The subroutine rl_filename_list_lcl() is a slightly modified copy of readline::rl_filename_list(). I removed all the decoration except ‘/’ for dirs, and I removed the check of $var_CompleteAddsuffix before decorating (I want it to happen unconditionally).

That solved the decoration issue, but not the “adds a space after every completion” issue. This is controlled by $rl_completer_terminator_character being unconditionally set to ‘ ‘ in readline.pm’s complete_internal(). I experimented with setting it to '' unconditionally, which did fix my problem. However, I didn’t want to leave it like that unconditionally for fear of unexpected side effects. Then I came across this in the code:

##   $rl_completer_terminator_character -- what to insert to separate
##      a completed token from the rest.  Reset at beginning of
##      completion to ' ' so completion function can change it.

Perfect! I’m already defining a custom function, so all I have to do is set $readline::rl_completer_terminator_character to '' in it.

Et voila! My completed toy function with auto-complete behaving the way I want it to:

$| = 1;
use Term::ReadLine;
my $Term = new Term::ReadLine 'Installer';

# tell Term::ReadLine::Perl to use my filename expander for completion
$readline::rl_completion_function = "main::rl_filename_list_lcl";

my $file = $Term->readline('Enter a file path: ');
print "$file = '$file'n";

exit;

sub rl_filename_list_lcl {
    my $pattern = $_[0];
    my @files = (<$pattern*>);
    foreach (@files) {
        if (-d $_) {
            $_ .= '/';
        }
    }

    # The readline.pm library sets this to ' ' on every completion call, we can
    # force it to '' here but have to do it every time
    $readline::rl_completer_terminator_character = '';

    return @files;
}

We’ve been loving our Trac install at work, but we are still sanding down a few rough spots.  One particular spot that’s been bobbing up has been what happens when “new” tickets have an owner assigned.  I liked the ticket status being “new”, because it meant that I could easily look at my ticket list and see tickets that had been directly assigned to me that I hadn’t acknowledged yet by accepting them.  Someone else really disliked having to “accept” a ticket that was already assigned to him before he could move it to QA.

I actually managed to create a solution for this using the TicketConditionalCreationStatusPlugin from TracHacks.  This plugin allowed me to conditionally set the status of the ticket based on the contents of the “owner” field.  This was actually a pretty slick solution, but for various reasons it didn’t sit well with a lot of people.

The next idea we had was to prevent new tickets from being created with any owner besides the default value of “dev” (our dev team mailing list).  This solution had three main benefits – It worked the same for everyone; no one would ever have a “new” ticket assigned to them; and the “dev” alias was always associated with a new ticket, which has some value in relation to email notifications.

Easy enough, right?  Maybe it’s just me, but I could not find a good way to enforce this.  There’s no native ability to hide fields that aren’t drop-downs, and owner’s not a drop down by default.  It can be made a drop-down, but not in a manner which would allow it to be removed.  There are several owner-oriented plugins, but none of them differentiate between the “new ticket” screen and the “modify ticket” screen.

After a bit of fruitless experimentation I realized that I was overthinking it.  I made the following change:

- <body py:match="body" py:attrs="select('@*')">
+ <body onload='document.getElementById("field-owner").disabled = true;' py:match="body" py:attrs="select('@*')">

et voila!  My default owner value is visible but immutable!

I know this isn’t a rock-solid solution, but in our environment (restricted corporate access) I’m not worried about evil-doers and griefers, I’m just wanting to remind people that they’re not supposed to change the value…