What Changed in the New Version of Neal Stephenson’s Reamde?

Last night I received an email from Amazon stating that the copy of Neal Stephenson’s Reamde was “missing content” and that there was a new version available.  Specifically:

We’re writing about your past Kindle purchase of Reamde: A Novel by Neal Stephenson. The version you received had Missing Content that have been corrected.

An updated version of Reamde: A Novel (ASIN:B004XVN0WW) is now available. It’s important to note that when we send you the updated version, you will no longer be able to view any highlights, bookmarks, and notes made in your current version and your furthest reading location will be lost.

I didn’t deal with it right away because I knew I had at least one highlight that I wanted to save.

This morning, it occurred to me that, since I’m 14% into the book, if I keep reading from the same spot I might never see the “missing content”.  Since I still had the old version, I decided to make a copy of it and see if I could figure out what changes they were delivering to me.  I saved my current position and the highlight, then responded to Amazon’s email to get the new copy.

While waiting for the new copy, I googled around a bit to see if I could details of what changed elsewhere.  I didn’t turn up those details, but I did find a few sites noting that the book had been pulled from Amazon (it’s back now), as well as lots of complaints about formatting issues.  I hadn’t noticed those issues myself.

After I got the new copy, I ninja’d up a text copy of both the old and new version, then massaged them a bit to make diffing easier.  Specifically I deleted blank lines, and then made each sentence its own line.

As expected, the vast majority of the changes were formatting.  The most common change was the removal of an odd character at the beginning of some words.  The character is 0xC2AD.  It renders as a dash in some tools, and I think it was probably not rendered at all in the Kindle software.

The rest of the punctuation changes were the addition and removal of dashes (“sledge-hammer” became “sledgehammer”), the occasional smart quote fix, and moving some commas around.  I have the original diff, but I’m not sure how interesting it is so I’m not posting it.

Now for the meat – were there any meaningful changes to the text?  There were indeed, though nothing so dramatic as a missing chapter.  Here are all the meaningful changes that I could find:

Two sentences were elided and combined in first version:

- Details of history that Geraldine and herm it was like a dog whistle.”
+ Details of history that Geraldine and her staff wouldn’t necessarily put together into a bigger picture.
+ But his fans—to them it was like a dog whistle.”

This entire sentence was missing from the original.  previous and next sentences provided for context:

  And if there is some other such thing I have not heard of yet—none of that either!”
+ In Russian, Csongor said, “What if we need to go into the world of T’Rain?”
  In English, Ivanov said, “Only exception to rule: Zula can play T’Rain if necessary.

Another example of sentences being combined:

- Probing been preparing for the Wor one month in advance, who’s to say they weren’t preparing for it six months or even twelve in advance?”
+ Probing attacks on what would soon become the Earthtone Coalition’s front lines.
+ Which leads to the question, if certain people had been been preparing for the Wor one month in advance, who’s to say they weren’t preparing for it six months or even twelve in advance?”

I’m not really sure if this is a meaningful change or not, but it’s certainly bigger than punctuation:

- “Yes, but—”to do with each
+ “Yes, but—”

Some missing words from the original:

- She called you at 8:42 and told you this story about REAMDE investigation and said she needed to know who had cast a healing spell on her character.”
+ She called you at 8:42 and told you this story about working with me on the REAMDE investigation and said she needed to know who had cast a healing spell on her character.”

Here’s a change that’s not especially meaningful but I’m including because it’s actually a new typo, which is interesting:

- The thin smile came back. “I believe we have that adequately covered.”
+ The thin smile came back. “I believe we have that adequately covered.”1P

Last one – never would have gotten this far if I hadn’t been doing this, but it looks like they messed up the cover design credit.  Oops!

- Cover design by James Iacobelli
+ Cover design by James Lacobelli

And that is that – nice to have a cleaned up version, but nothing to worry about missing if you’ve already started the book.

(Thanks to Perl, diff, and WinMerge)

It’s The Little Things That Get Me

My website is hosted by Dreamhost, currently on a server named “tucana” (previously on “hi-c”, RIP).  Two weeks ago tucana had a couple of outages.  These were definitely longer than I’m used to, but I more than get what I pay for with Dreamhost – if I were selling services I would be using a coloed server.  For the money I pay, a 24 hour outage every 2-3 years is actually pretty good.

So I wasn’t too terribly put out by the outage – I had local backups from the night before.  I sat back to wait it out.  It came back up with (mostly) all of my files(*) and I was happy.  But they made one small change that I noticed immediately – they changed the “font” of the server name in the motd file.

Here’s the old banner:

_                         
| |_ _  _ __ __ _ _ _  __ _
|  _| || / _/ _` | ' \/ _` |
 \__|\_,_\__\__,_|_||_\__,_|
                             
Welcome to tucana.dreamhost.com

And here’s the new banner:

|                            
  _| |  |  _|  _` |   \   _` |
\__|\_,_|\__|\__,_|_| _|\__,_|
                                
Welcome to tucana.dreamhost.com

I noted it and then moved on, but my brain can’t let it go.  Every time I log in I notice that the banner is different.  I’m really ready for my brain to stop caring about this…

(*) Everything in my home directory was found and current, but, since they put a new OS on the server, my crontab was missing.  They eventually restored one, but it was several weeks old.  Fortunately at some point I started dumping it into my home directly every night so it would get picked up by my backups – whew!

Setting Default Font in TestLink

When I first set up TestLink, one of my first actions was to figure out how to force FCKEditor to use a monospace font. Much of our testing takes place in terminals using a monospace font, and I thought it would be easier to engage with the expected results if they were also in a monospace font. This was a fairly easy thing to do using the TestLink and FCKEditor docs, but since I haven’t written it down before, here are the steps using a monospace font by default in TestLink using FCKEditor.

Update 2012-05-08: There is a usable-with-the-patch-command patch available that incorporates all of these changes and more. See the README at the base of the github repo for more details on what’s in that patch.

First, create a file called custom_textarea.css. I placed this in the root of my install, which probably wasn’t the best place for it. The contents are a lightly edited copy of body class from gui/themes/default/css/testlink.css:

custom_textarea.css
1
2
3
4
5
6
7
body {
    background:     #FFF;
    font-family:    'Courier New', 'Trebuchet MS', Arial, Verdana, sans-serif;
    font-size:      small;
    margin:         0px;
    padding:        0px;
}

The “light editing” I did was to push ‘Courier New’ onto the front of the font-family.  I also changed it so that the edit window background would be white instead of grey by changing the background color.

Then I told FCKEditor to use this CSS as the style for the edit boxes by adding the following line to cfg/tl_fckeditor_config.js:

cfg/tl_fckeditor_config.js
1
2
// Set our own custom css (sets white background and mono-space font)
FCKConfig.EditorAreaCSS = FCKConfig.BasePath + '../../../custom_textarea.css';

This did accomplish the task that I had set for myself, which was changing the default font displayed by FCKEditor when editing text.  However, I also assumed that it would, when the text in the editor was saved, cause that saved text to be displayed in a monospace font.  It turns out that this didn’t work though.  I can’t decide if this is a bug in TestLink or not, but these defaults don’t get applied when the text is displayed.

I spent quite a bit of time looking for the correct knob to fix this in the TestLink docs and the FCKEditor docs, and I couldn’t find one.  Font editing works, but you have to explicitly change the font option while editing to make it work.

What I ended up doing was to edit the two Smarty templates used for displaying steps so that each action and step were wrapped in a div I could control, and then changed the style for that div so that the default font was monospaced.  In a way this was actually a better solution than a general FCKEditor knob because it allowed me to just change the default font in action/expected results, and not other text, like Summary, that might look better as a variable-width font.

To achieve this I first added two new classes to gui/themes/default/css/custom.css.  This could have been done with just one class, but I liked the idea of being able to change the style of actions and results independently in the future.

gui/themes/default/css/custom.css
1
2
3
div.c_actions, div.c_expected_results {
    font-family: 'Courier New', Courier, monospace;
}

Then I added the class to inc_steps.tpl and tcStepEdit.tpl.

gui/templates/testcases/inc_steps.tpl
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
--- inc_steps.tpl.dist  2011-08-22 21:08:21.000000000 -0500
+++ inc_steps.tpl       2011-09-26 09:42:25.000000000 -0500
@@ -53,10 +53,10 @@
                                size="{#STEP_NUMBER_SIZE#}"
                                maxlength="{#STEP_NUMBER_MAXLEN#}">
                        {include file="error_icon.tpl" field="step_number"}
                        </span>{$step_info.step_number}
                </td>
-               <td {if $edit_enabled} style="cursor:pointer;" onclick="launchEditStep({$step_info.id})" {/if}>{$step_info.actions}</td>
-               <td {if $edit_enabled} style="cursor:pointer;" onclick="launchEditStep({$step_info.id})" {/if}>{$step_info.expected_results}</td>
+               <td {if $edit_enabled} style="cursor:pointer;" onclick="launchEditStep({$step_info.id})" {/if}><div class="c_actions">{$step_info.actions}</div></td>
+               <td {if $edit_enabled} style="cursor:pointer;" onclick="launchEditStep({$step_info.id})" {/if}><div class="c_expected_results">{$step_info.expected_results}</div></td>
                {if $session['testprojectOptions']->automationEnabled}
                <td {if $edit_enabled} style="cursor:pointer;" onclick="launchEditStep({$step_info.id})" {/if}>{$gui->execution_types[$step_info.execution_type]}</td>
                {/if}
gui/templates/testcases/tcStepEdit.tpl
1
2
3
4
5
6
7
8
9
10
11
12
13
--- tcStepEdit.tpl.dist 2011-09-22 10:27:23.000000000 -0500
+++ tcStepEdit.tpl      2011-09-26 09:43:47.000000000 -0500
@@ -211,8 +211,8 @@
        {/if}
       {else}
         <td style="text-align:left;"><a href="{$hrefEditStep}{$step_info.id}">{$step_info.step_number}</a></td>
-               <td ><a href="{$hrefEditStep}{$step_info.id}">{$step_info.actions}</a></td>
-               <td ><a href="{$hrefEditStep}{$step_info.id}">{$step_info.expected_results}</a></td>
+               <td ><a href="{$hrefEditStep}{$step_info.id}"><div class="c_actions">{$step_info.actions}</div></a></td>
+               <td ><a href="{$hrefEditStep}{$step_info.id}"><div class="c_expected_results">{$step_info.expected_results}</div></a></td>
         {if $session['testprojectOptions']->automationEnabled}
                  <td><a href="{$hrefEditStep}{$step_info.id}">{$gui->execution_types[$step_info.execution_type]}</a></td>
                {/if}

These aren’t exactly what my diffs look like since I removed the links from the display text last week, but I believe it will work.

Better Copy/Paste When Editing Test Steps in TestLink

My current biggest peeve with TestLink is how hard TestLink makes it to copy text out of test steps when editing a test case. Both when viewing a case and when editing test case steps via the Test Specification interface, clicking on the text of a test step causes you to start editing the text in that step.

I find this behavior very off-putting. I constantly find myself trying to copy in these views, either to copy repetitive actions from a previous step, or to do a dry-run executing a test case. I finally got mad enough today to spend the time figuring out how to change this behavior.

Update 2012-05-08: There is a usable-with-the-patch-command patch available that incorporates all of these changes and more. See the README at the base of the github repo for more details on what’s in that patch.

There are two Smarty templates that control the layout of these views, both of which live in gui/templates/testcases/. inc_steps.tpl controls the layout when viewing a test case (but not editing steps), and tcStepEdit.tpl controls the layout when editing steps.

After applying these patches, you can copy text out of the Step Action and Expected Results columns. To edit the row as before the patch, click on the row counter (the first cell in the row).

These patches are against TestLink 1.9.3

inc_steps.tpl
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
--- inc_steps.tpl.dist  2011-08-22 21:08:21.000000000 -0500
+++ inc_steps.tpl       2011-09-22 10:08:55.000000000 -0500
@@ -53,10 +53,10 @@
                                size="{#STEP_NUMBER_SIZE#}"
                                maxlength="{#STEP_NUMBER_MAXLEN#}">
                        {include file="error_icon.tpl" field="step_number"}
-                       </span>{$step_info.step_number}
+                       </span><div {if $edit_enabled} style="cursor:pointer;" onclick="launchEditStep({$step_info.id})" {/if}>{$step_info.step_number}</div>
                </td>
-               <td {if $edit_enabled} style="cursor:pointer;" onclick="launchEditStep({$step_info.id})" {/if}>{$step_info.actions}</td>
-               <td {if $edit_enabled} style="cursor:pointer;" onclick="launchEditStep({$step_info.id})" {/if}>{$step_info.expected_results}</td>
+               <td>{$step_info.actions}</td>
+               <td>{$step_info.expected_results}</td>
                {if $session['testprojectOptions']->automationEnabled}
                <td {if $edit_enabled} style="cursor:pointer;" onclick="launchEditStep({$step_info.id})" {/if}>{$gui->execution_types[$step_info.execution_type]}</td>
                {/if}
tcStepEdit.tpl
1
2
3
4
5
6
7
8
9
10
11
12
13
--- tcStepEdit.tpl.dist 2011-09-22 10:27:23.000000000 -0500
+++ tcStepEdit.tpl      2011-09-22 10:39:55.000000000 -0500
@@ -211,8 +211,8 @@
        {/if}
       {else}
         <td style="text-align:left;"><a href="{$hrefEditStep}{$step_info.id}">{$step_info.step_number}</a></td>
-               <td ><a href="{$hrefEditStep}{$step_info.id}">{$step_info.actions}</a></td>
-               <td ><a href="{$hrefEditStep}{$step_info.id}">{$step_info.expected_results}</a></td>
+               <td >{$step_info.actions}</td>
+               <td >{$step_info.expected_results}</td>
         {if $session['testprojectOptions']->automationEnabled}
                  <td><a href="{$hrefEditStep}{$step_info.id}">{$gui->execution_types[$step_info.execution_type]}</a></td>
                {/if}

I’ve been writing test cases with this change in place all morning and it is much, much nicer than the default behavior

I Think My Plumber’s A System Administrator…

My house’s water is provided by a well, and I live in a notoriously low-water part of my county. As such, every year since I’ve lived here I’ve had water issues in the fall. I have a “well guy” who is helping me with these issues, diagnosing minor issues, delivering water for the well when there’s no other alternative, etc.

This guy’s great, but I always feel on the back foot around him and I’ve never really known why. He came out today to look into my current issue (actual leak in the line between well and house) and I myself trying to dive deeper into what strikes me as odd about him. Here’s my list:

  1. Rarely rude, but often so overworked that he’s very brusque.
  2. Very obviously cares more about fixing my problem because it offends him that the problem exists, not because it affects me, the customer, in any way.
  3. Often proceeds with fixes without adequately explaining the exact nature of the problem. I often have to drag post mortems out of him later.

So then it hits me – this guy’s a sys admin! He’s sharp, he has an intuitive grasp of troubleshooting in his domain, and he cares more about the systems in his domain than the people affected! How have I never noticed this before?

Using Wiki Markup in Trac Custom Ticket Fields

Yesterday I worked on streamlining our ticket workflow in Trac. While I was in the test project anyway, I decided to address an issue that had bugged me since I first set up Trac. I originally set up two custom ticket fields related to case IDs in our CRM system:

crmnum = text
crmnum.label = Case #
crmnum.format = plain
crmnum.order = 1
crmlink = text
crmlink.label = Case Link
crmlink.format = wiki
crmlink.order = 3

The idea was that we could have the “crmnum” field to just hold the ID, which we could act on programmatically. The “crmlink” field would be a copy/paste link of the ticket’s URL, which would allow someone looking at the ticket to click through to the CRM case easily.

The problem with this is that the link is long and ugly and completely throws off the tidiness of the ticket summary.

So, having recently figured out how to write a custom wiki macro, I decided to create one for links to our CRM system also. The idea would be that the “crmlink” field would go away and the “crmnum” field could be updated to use the wiki Macro to generate the link automatically.

I nuked the “crmlink” field and changed “crmnum” from “type = plain” to “type = wiki“. I wrote my custom CRM macro, testing it in the wiki itself until I had it behaving. I then created a new ticket, entered [[CRM(123)]] in the crmnum field, and clicked “Preview”. Instead of seeing the link it should have returned, instead I got [[CRM(...)]]. I did some testing and proved to myself that some wiki processing was taking place (ticket:1 expanded correctly) and that no macro expanded in that field, including built-in macros.

Then on a hunch, I changed crmnum to “type = textarea” instead of “type = text“. Sure enough, macros work properly in textarea fields but not text fields. In a way this makes sense since almost all built-in macros return multiple lines of text. The documentation for custom ticket fields doesn’t mention that wiki processing behavior differs between these field types. Unfortunately the textarea fields can’t easily be tricked into looking like a simple text field. They come with an editing bar, and they are assigned the same class as the description field, so you can’t easily use CSS tricks on it.

I was unsure if this was a bug or intentional, but I managed to find the original Trac ticket for adding wiki processing to custom fields, and it’s clear that it was intentional to make the wiki-processing behavior of the two field types differ. Specifically, text fields are processed with format_to_oneliner() and textareas are processed with format_to_html() (this can be seen in the change to web_ui.py associated with new feature.

After figuring out that the text/textarea parsing differences were done intentionally, I lost interest in patching the code itself. Instead I looked into using InterWiki links instead of macros, which are processed in text fields.

I created trac ticket 10352 about this issue, hopefully it gets addressed one way or another.

Linking to TestLink testcases from Trac addendum

Last week I posted about a custom [[TestLink()]] macro I had written for Trac. Yesterday I wrote another macro to do something similar, but in doing so I found out that I was probably doing a lot more work than I needed to.

The macro I wrote allowed me to write [[TestLink(m3-123)]] to link directly to test case 123 in the m3 project. It was about seven lines of python and not the most complex thing in the world, but nowhere near as simple as using an InterWiki prefix. I did this by adding the following line to my Trac install’s InterMapTxt wiki page:

TestCase https://devtest.example.com/linkto.php?tprojectPrefix=$1&item=testcase&id=$1-$2

This allows me to replace [[TestLink(m3-123)]] with testcase:m3:123.

There are tradeoffs of course. The macro allows full control using python. That allowed me to write a single macro that could produce links to different places in our CRM system based on the format of the single argument to the macro. Since I don’t get that flexibility with InterWiki links, I had to use two InterWiki links instead of a single macro.

On the other hand, the InterWiki links are simpler to write, look better, and are much closer to the standard Trac wiki link style that my users are already used to. Also, they have the advantage that they are expanded in ticket custom text fields and macros aren’t.

In the end I’ll keep both the macros and the InterWiki links for these cases since they aren’t mutually exclusive, but I’ll definitely turn to InterWiki links before I turn to writing a custom macro in the future.

Arduino’s map() Function and Numeric Distribution

The Arduino map() function is an interesting beast. Very technically it works exactly as its documented to work, but not the way almost every example uses it.

Here’s an example you can find in hundreds of sketches online, including the actual documentation for map():

val = map(val, 0, 1023, 0, 255);

This is a simple map, and one would expect that every four ticks on the input would map to one tick on the output (that is, {0,1,2,3} -> 0, {4,5,6,7} -> 1, etc). But that’s not what the function above actually does.

To show the issue, let’s make the output range smaller (but still an even divisor):

val = map(val, 0, 1023, 0, 15);

This should result in an even distribution, 64 input ticks per one output tick. To test this, I wrote a quick script implementing the Arduino map logic (which they were nice enough to document):

long map(long x, long in_min, long in_max, long out_min, long out_max)
{
  return (x - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
}

The script prints a table of output values and the number of times that value was returned. Here’s the output for map(?, 0, 1023, 0, 15) for each value in 0..1023:

map(0..1023, 0, 1023, 0, 15);
  0   69
  1   68
  2   68
  3   68
  4   68
  5   69
  6   68
  7   68
  8   68
  9   68
 10   69
 11   68
 12   68
 13   68
 14   68
 15    1

That’s definitely not an even distribution. Now here’s a really stark example:

map(0..1023, 0, 1023, 0, 1);
  0 1023
  1    1

That’s a pretty egregious imbalance.

I mentioned earlier that the function’s actually working how it’s documented to work, just not how it’s usually used in examples. The map() docs state that “[t]he map() function uses integer math so will not generate fractions, when the math might indicate that it should do so. Fractional remainders are truncated, and are not rounded or averaged.”

This completely makes sense – if you imagine a range of 1024 values between 0 and one, all of them will be less than 1 except the last value, and since it’s integer arithmetic, all the less-than-1 values are 0.

The solution is fairly simple – increase the in_max and out_max args by one more than the actual maximum value (and then wrap the output in constrain(), which you ought to have done anyway). It’s fairly easy to work through why this works in your head, but here are the same examples I gave above with the increased maximums:

map(0..1023, 0, 1024, 0, 2);
  0   512
  1   512
map(0..1023, 0, 1024, 0, 16);
  0   64
  1   64
  2   64
  3   64
  4   64
  5   64
  6   64
  7   64
  8   64
  9   64
 10   64
 11   64
 12   64
 13   64
 14   64
 15   64

I have worked through this and I now understand how to get the values I want out of constrain. What I don’t know, however, is why the docs and the examples don’t address this issue. I know I’m not the first person to find this issue because I’ve found sketches on the internet where people are doing this (and they’re doing it in a way that leads me to believe they’re doing it on purpose), but I couldn’t get Google to show me any pages where someone actually addresses this point. I wonder why?

As for why I was looking into this, I was playing around with a couple of pots and an LCD, making a fakey etch-a-sketch using the 2-row LCD I have for such things. I could not figure out why my vertical control would only drop me down a line when I had the pot turned 100%. Now I know…

Edit 2011-09-13:

I posted this in the arduino.cc forum last night. As I expected, this is an issue known in the arduino community. Among other things, I was referenced to this post discussing the issue. My problem with all of this isn’t so much that it’s odd behavior, but that it’s been odd for at least 2 years and there’s no mention of the oddness in the map() documentation. The docs are a wiki, so I proposed an addition in the forum thread, maybe I’ll get to add it and help someone out in the future.

Custom TestLink Macro for Trac

I’ve been spending a lot of time lately trying to get the workflow between Trac and TestLink to make sense. I’m far from having the perfect solution, but a great first step was creating a Trac wiki macro that will link to specific test cases in TestLink. There’s lots of areas in which this could be improved in the future…

Create this as conf/TestLinkMacro.py in your project (replacing test.example.com with your TestLink host):

conf/TestLinkMacro.py
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
import re
 
from trac.wiki.macros import WikiMacroBase
 
class TestLinkMacro(WikiMacroBase):
    """Macro to generate automatic links to test cases in TestLink.
    """
 
    def expand_macro(self, formatter, name, args):
        """Expand a TestLink test case ID into a link
        """
        TestLinkServer = 'test.example.com'
        matches = re.match(r'(.*)-(d+)$', args)
         
        return '<a href="https://' + TestLinkServer + '/linkto.php?tprojectPrefix=' + matches.group(1) + '&item=testcase&id=' + matches.group(0) + '">[[TestLink(' + matches.group(0) + ')]]</a>'

After putting this in place, and possibly restarting Trac, you can create a direct link to test case m3-123 in a wiki by typing [[TestLink(m3-123)]]. Note that I could have changed the actual string displayed in the wiki to anything I wanted, but I chose to leave it looking identical to the actual macro (but linked) so that it would reinforce to people the correct format for the test case links.

This macro is useful in several ways, but the most obvious is allowing us to enter regression test case IDs into commit messages (which then get associated with a ticket in Trac and are displayed with the wiki engine, making the test case link clickable).
With commit messages being associated with tickets automatically and being displayed by the wiki engine, this gives us a nice syntax to enter regression case links into commit messages and be very followable later

Edit 2011-09-13:

See also this update to my use of macros for linking to test cases and this comment on custom ticket field wiki parsing, both of which have a direct relation to this post

This is a Thing I Made

I made this a while ago.

It’s 8 LEDs and a pot, with the position of the pot controlling how many of the LEDs are lit. This is not a complex project, but I like it very much because it’s the first thing I put together that was not just copied from a tutorial. The shift register/8 LEDs was from one project, but I figured out how to add a pot to the circuit, read the pot position off of an analog pin, and set the LEDs accordingly. I am very proud of my silly little circuit.