I have finally come to terms with the TestLink API and have bashed out an automated test runner written in perl (which I hope to clean up and post in the next few days, there don’t seem to be many (or any) examples of this in the wild).

After getting nightly automated testing running, I wanted a very simple script which would, for each test case in a given test plan, compare the last and the second to last result and print a warning if they were different. Long term I will have better automated reporting, but our test script library is still so new that it’s not really news that a lot of them are failing. What would be news is if a specific test case passed two nights ago but failed last night.

This seemed like a really easy thing to do, but I couldn’t find an API function to support it. The closest thing I could find was getLastExecutionResult(), but that only gave me the last execution, it didn’t give me the execution history I needed. I also could have leveraged the fact that getTestCasesForTestPlan() includes each case’s last execution status to have done in-place alarming if the new status didn’t match the old status, but I felt like that would limit my ability to expand reporting in the future. I toyed with the idea of getting the info out of the DB directly, but decided instead of just implement the API function I needed. It’s mostly a copy of getLastExecutionResult(), but it has an added parameter numexecs which allows you to specify how many executions to return.

I hope to wrap all of these TestLink patches up into a better place at some point in the future, but for now it feels better to just get the code out where Google can find it as fast as possible.

Here’s the patch for adding the getExecutionResults() function:

Update 2012-05-08: There’s a real, usable-with-patch-command patch file for this change available at the github repo I started for my TestLink changes.

--- xmlrpc.class.php	(revision 268)
+++ xmlrpc.class.php	(revision 270)
@@ -176,6 +176,7 @@
 	public static $testSuiteNameParamName = "testsuitename";
 	public static $timeStampParamName = "timestamp";
     public static $titleParamName = "title";
+    public static $numexecsParamName = "numexecs";
 
 
     public static $urgencyParamName = "urgency";
@@ -255,6 +256,7 @@
 	                            'tl.getBuildsForTestPlan' => 'this:getBuildsForTestPlan',
 	                            'tl.getLatestBuildForTestPlan' => 'this:getLatestBuildForTestPlan',	
                                 'tl.getLastExecutionResult' => 'this:getLastExecutionResult',
+                                'tl.getExecutionResults' => 'this:getExecutionResults',
 	                            'tl.getTestSuitesForTestPlan' => 'this:getTestSuitesForTestPlan',
 	                            'tl.getTestSuitesForTestSuite' => 'this:getTestSuitesForTestSuite',
 	                            'tl.getTestCasesForTestSuite'	=> 'this:getTestCasesForTestSuite',
@@ -1220,8 +1222,78 @@
 
 		return $maxbuild;
 	}
-	
+
 	/**
+	 * Gets the result of LAST EXECUTION for a particular testcase 
+	 * on a test plan, but WITHOUT checking for a particular build
+	 *
+	 * @param struct $args
+	 * @param string $args["devKey"]
+	 * @param int $args["tplanid"]
+	 * @param int $args["testcaseid"]: optional, if does not is present           
+	 *                                 testcaseexternalid must be present
+	 *
+	 * @param int $args["testcaseexternalid"]: optional, if does not is present           
+	 *                                         testcaseid must be present
+	 * @param int $args["numexecs"]: optional.  If set, number of cases to
+	 *                               return.  If unset, all will be returned
+	 *
+	 * @return mixed $resultInfo
+	 *               if execution found, array with these keys:
+	 *               id (execution id),build_id,tester_id,execution_ts,
+	 *               status,testplan_id,tcversion_id,tcversion_number,
+	 *               execution_type,notes.
+	 *
+	 *               if test case has not been execute,
+	 *               array('id' => -1)
+	 *
+	 * @access public
+	 */
+	public function getExecutionResults($args)
+	{
+		$operation=__FUNCTION__;
+		$msg_prefix="({$operation}) - ";
+		
+		$this->_setArgs($args);
+		$resultInfo = array();
+		$status_ok=true;
+		
+		// Checks are done in order
+		$checkFunctions = array('authenticate','checkTestPlanID','checkTestCaseIdentity');
+		
+		$status_ok=$this->_runChecks($checkFunctions,$msg_prefix) && 
+		           $this->_checkTCIDAndTPIDValid(null,$msg_prefix) &&
+		           $this->userHasRight("mgt_view_tc");       
+		
+		if( $status_ok )
+		{
+			// get all, then return last
+			$sql = " SELECT * FROM {$this->tables['executions']} " .
+			       " WHERE testplan_id = {$this->args[self::$testPlanIDParamName]} " .
+			       " AND tcversion_id IN (" .
+			       " SELECT id FROM {$this->tables['nodes_hierarchy']} " .
+			       " WHERE parent_id = {$this->args[self::$testCaseIDParamName]})" .
+			       " ORDER BY id DESC";
+
+			$limit   = $this->args[self::$numexecsParamName] > 0 ? $this->args[self::$numexecsParamName] : -1;
+			$records = $this->dbObj->get_recordset($sql, null, $limit);
+			
+			if(null == $records)
+			{
+				// has not been executed
+				// execution id = -1 => test case has not been runned.
+				$resultInfo[]=array('id' => -1);
+			} 
+			else
+			{
+				$resultInfo = $records;
+			}
+		}
+		
+		return $status_ok ? $resultInfo : $this->errors;
+	}
+
+	/**
      * Gets the result of LAST EXECUTION for a particular testcase 
      * on a test plan, but WITHOUT checking for a particular build
      *

This function suffers from a few warts, but they are ones mostly inherited from getLastExecutionResult(). If I were to polish this up (beyond my normal TestLink API beefs like some sort of standardized return format), I would definitely add optional buildid and platformid parameters so you could narrow down the executions you’re looking for even more. But that’s a problem for another day…