Along with JUnit, JWebUnit, NUnit, and SimpleTest, cfix was one of the nominees for the Automated Testing Institute’s Automation Honors Award 2009 in the category Best Open Source Unit Automated Test Tool. A few days ago, the results were published and cfix finished second — surpassed only by JUnit, which finished 1st (No real surprise here). If you are interested, there is a video in which the results are presented.

Jeff Wilcox has described in this article how to make NUnit tests run in the Silverlight unit testing framework. That article provides all the groundwork for what you need to do:

• Compile a version of the NUnit framework that runs in the Silverlight 3 context
• Provide metadata for the Silverlight unit testing framework so it knows which attributes to look for in order to locate what method is a test (and a little more)
• Register this metadata in your Silverlight unit test project.

I’d like to thank Jeff for his great work, and provide some raisins on top: an update to version 2.53 of NUnit. Here’s what’s to do:

• Take the compatibility shim and test metadata provider from Jeff’s article.
• Download the current NUnit source code (in this case 2.5.3). Create a new Silverlight project, and add the code for the NUnit framework into it. Compile: a whole bunch of exceptions come up.
• NUnit’s source code uses the #if precompiler directive in order to exclude parts that are incompatible with different version of the runtime. By defining the NET_2_0 symbols, we can make a lot of the errors disappear: the nongeneric implementation of Stack, for example, is excluded now. We need to exclude some more code: Instead of deleting it altogether, I chose to use the SILVERLIGHT symbol and add a #if !SILVERLIGHT directive to get rid of unsupported code instead. In cases where a whole method body is excluded, add a section on the bottom that throws a NotSupportedException:

  #if SILVERLIGHT
        throw new NotSupportedException( "Not supported in Silverlight" );
  #endif
  

• We need to get rid of the serialization constructors from all the exceptions.
• Remove the attributes for threading demands – they’re useless in Silverlight anyway.
• Remove the attributes for threading demands – they’re useless in Silverlight anyway.
• Add a new shim class for the non-generic System.Collections.Specialized.ListDictionary to the Compatibility project. It is a simple derivation of the genetic Dictionary.
• In the PathComparison class, there are some unsupported string operations. I didin’t bother exluding and replacing these, and just refactored them to supported alternatives. (Thinking about it, I should probably have used the .NET2.0 switch. Next time, promised)
• Finally, the metadata provider uses a changed property for in the ExpectedException attribute. Corrected, aaaaaand…
• Done. Now all you need to do is reference the three generated assemblies, follow Jeff’s article on how to set up the testing framework for the NUnit provider, and start writing tests.

This was easy enough that I hope it can be done for any upcoming version of NUnit.

DOWNLOAD SOURCE CODE
DOWNLOAD BINARIES

The point where I’d like to disagree with Jeff Wilcox the idea that NUnit was a legacy solution, and that you should start new projects with Visual Studio testing. NUnit provides much richer assertions, and a better syntax for setting them up: you won’t find assertions for examining collection content or thrown exceptions in Visual Studio testing. Or the upcoming ability to run tests in parallel compartments to simulate a distributed environment. NUnit is far from being a discontinued legacy solution, it is the de facto standard and quickly developing. And: thanks to the extensibility of the Silverlight unit testing framework, there’s not a reason not to use it for Silverlight as well.

Software Development Times has just published Guest View: Wrestling with Scaling Software Agility by Ryan Martens and me. This Guest View is a little unique in that Ryan and I actually try to wrestle each other to the ground… Here is why we try to do so:

Agile champions spend a lot of time trying to communicate the agile premise to the executives in their organization. The difference in context between the champion and the executive often makes it a difficult conversation. A Scrum Master versed in behavior-driven design is not always able to relate to the frustrations of a sales executive who gets free advice on how to sell from everyone and his grandmother.

Conversely, a CFO does not necessarily understand why unit test coverage on the company’s legacy code is still inadequate after a full year of investment in agile methods that embrace refactoring as a core practice.

To bridge the chasm through this article, we resort to role-playing. Ryan Martens plays the Agile Champion; Israel Gat plays the Skeptical Executive. Metaphorically speaking, each one tries to wrestle the other to the ground.

Before you get into this Guest View, I would like to reinforce an important disclaimer:

A note of caution before Ryan and Israel make irreparable damage to their long-standing relationship: The two actually understand each other extremely well and rarely are they of different opinions on the fundamentals of agile in real life…

Enjoy the article!

I have just started to play with PHPUnit and I can easily say that this is a must have tool for every PHP developer.

Quoting Matthew Weiner O’Phinney:

“If you are not doing unit-testing, then you are not a good developer.”

This might sound harsh but is in fact a bit of a truth since most of the developers don’t really mind or have the time to go on testing the scripts in a block or whole application overview.

Developing the scripts for PHPUnit are not hard, even though I’m not an expert on the area, it don’t take too much time to develop a good script to test your scripts.

The tests are based on assertions and mock-ups from how your script should work. It will basically create a very least minimal requirement to see if your script is working.

Let’s say, if your object have to set 3 properties, call a method and then another object should load this just created object, the test script should cover all of this.

Afterwards is just run, at command prompt, phpunit yourScript.php and check for the results.

As I have mentioned before, I’m no expert since I have just started to learn PHPUnit, so I’m going to keep this post small until I can create a small tutorial of how to work with PHPUnit but meanwhile:

To install:

Install it using PEAR – It’s the easier way. If the install fails because PEAR is out of date, then run pear upgrade-all to update pear and all packages installed.

Check out Chapter 3 of PHPUnit online documentation to learn how to fully install it: http://www.phpunit.de/manual/3.0/en/installation.html

To write tests:

Check out Chapter 4 to learn the basics of writting tests to run PHPUnit. The online documentation is not big and it’s worth to read it fully. Anyway keep the Chapter 20, about the API, in your perspective to really see how PHPUnit can help you to make your scripts even stronger.

After giving a co-worker a hard time for not using TDD to create his Android app, I realised I was doing the same thing on my Griffon app. Unfortunately, I had a hard time testing the “withSql” closure and wanted a way to verify my sql statements were legit. Groovy metaClass came in handy again.

Method under test in my Controller
def downloadHorses = { horse ->
  withSql { sql ->
    sql.execute("insert into horse (id, name) values (?,?)", [it.id, it.name])
  }
}

Corresponding GroovyTestCase method
void testDownloadHorses() {
  model.availableHorses << new HorseModel(id: "1", name: "Horse 1")
  def fake = new FakeWithSql()
  controller.metaClass.withSql = fake.withSql
  controller.downloadCourses()
  assertEquals([1, "Horse 1"], fake.params)
  assertEquals("insert into horse (id, name) values (?,?)", fake.statement)
}

FakeController class that returns the sql statement and parameters.
class FakeWithSql {
  def params
  def sql = new Expando()
  public withSql = {Closure closure ->
    sql.execute = {sql, parameters ->
        statement = sql
        params = parameters
      }
    closure(sql)
  }
}

As I think about it, I will run into problems with multiple sql statements (only keeps track of last) and select statements (returning a set of data). This works well enough for now, but I will look into using an actual GroovyMock class instead of the ExpandoClass

We have an immediate need for a SENIOR SHAREPOINT DEVELOPER/BUSINESS ANALYST with our client in FREMONT, CA. This is a CONTRACT POSITION for 6 months with possible extension.

The ideal candidate should be able to work with the business team to articulate functional requirements, priorities requirements in terms of business value, prepare technical designs that will accommodate the requirements, and configure /develop the requisite solutions.

• Work with business users to identify, prioritize, and document the requirements
• Work with the requirements and prepare an implementation plan to address the requirements
• Prepare technical design that will accommodate the identified requirements/priorities.
• Develop/program/configure the SharePoint solutions based on the documented requirements and technical design.
• Participate in all development lifecycle activities: unit test, systems test, data migration, integration.
• Experience as a business analyst and Sharepoint developer designing and building MOSS solutions
• Must have experience working with business users to identify and document business requirements related to content management/document management/collaboration needs
• Must have strong experience using Visual Studio/.NET/SharePoint Designer, including the design and implementation of custom web parts and other programs that leverage the SharePoint object model.

If you are interested, please send us your resume along with the hourly rate, contact number, and availability for a phone interview to raj@mindsource.com.

Für einen Unit-Test benötigt man meistens nur gewisse Teile eines “fremden” Objekts. Oft genügt einem eine fixe Anzahl definierter Rückgabewerte. Statt das man nun das “fremde” Objekt erzeugt und die Werte aus einer Datenbank holt, setzt man eine Testattrappe ein. Diese Attrappe liefert einem die definierten Rückgabewerte oder Exceptions, ohne dabei wirklich mit der Datenbank, dem Dateisystem oder dem Netzwerk zu sprechen. So eine Attrappe kann ein Mock oder Stub sein.

JMockit ist ein Framework für Java, das einem bei der Erstellung von Mocks und Stubs hilft. Für Java gibt es zahlreiche solcher Frameworks. JMockit hat aber einige besondere Fähigkeiten, was das Testen deutlich erleichtern kann.

Für AtaraxiS war ich in den letzten Tagen am erweitern der Unit-Tests. Die Testabdeckung war noch nicht überall so gut, wie ich diese haben will. Es fehlten vor allem noch die zahlreichen Catch-Blocke für die Exceptions.
(Wieso überhaupt Exceptions testen? AtaraxiS will die Verschlüsselung abstrahieren und reduziert die dabei möglichen Exceptions auf eine handlichere Anzahl. Diese Tests dienen der Kontrolle, ob auch nach dem anstehenden Umbau noch alles wie gewohnt läuft.)

	public ACDecryptInputStream (File inFile, SecretKey aesKey) throws IOException
	{
		try
		{
			// Parse the Header of the File
			AtaraxisHeaderParser ahParser = new AtaraxisHeaderParser(inFile);

			// When it contains a Header, then prepare all for CBC-Mode
			IvParameterSpec ivSpec = new IvParameterSpec(ahParser.getIV());

			fis = new FileInputStream(inFile);
			fis.skip(ahParser.bytesToSkip());

			cipher = Cipher.getInstance("AES/CBC/PKCS7Padding", "BC");
			cipher.init(Cipher.DECRYPT_MODE, aesKey, ivSpec);

			// Create the CipherInputStream
			decryptedInputStream = new CipherInputStream(fis, cipher);
		}
		catch (NoSuchAlgorithmException e)
		{
			LOGGER.error("NoSuchAlgorithmException", e);
			throw new IOException("NoSuchAlgorithmException");
		}
		.....
	}

Um dies zu testen, müsste die Klasse Cipher bei getInstance() eine NoSuchAlgorithmException werfen. Dies könnte ich durch eine Abstraktionsschicht zwischen unserem Code und dem von BouncyCastle machen. Das bedingt aber eine Änderung am Code. Hier wäre das zwar möglich, doch geht das nicht immer.

JMockit hat dafür eine passende Methode. Mittels Mockit.redefineMethods() kann eine Methode umgeleitet werden. Das angegebene Ziel kann genau das machen, was man von ihm erwartet. Um die Exception zu werfen, dient dieser Aufruf:

	@Test(expected=IOException.class)
	public void ACDecryptInputStream_NoSuchAlgorithmException() throws Exception
	{
		Mockit.redefineMethods(javax.crypto.Cipher.class, new Object() {
			@SuppressWarnings("unused")
			public final Cipher getInstance(String transformation,
                    Provider provider)
             throws NoSuchAlgorithmException,
                    NoSuchPaddingException
			{
				throw new NoSuchAlgorithmException();
			}
		});

		ACDecryptInputStream acI = new ACDecryptInputStream(testFile, key);
		fail("IOException missing for " + acI);
	}

Dies ist nur eine der unzähligen Möglichkeiten, die JMockit bietet. Nach dem gleichen Prinzip kann man beliebige Rückgabewerte zurückliefern. Wer mehr dazu wissen will, findet auf der Projektseite von JMockit eine sehr detaillierte Anleitung.

Here’s a three-for-one special for you: A post about implementing the Levenshtein string distance algorithm in Scala AND refactoring it from an imperative style to a functional style AND I even throw in a short lesson on memoization. To make sure that our refactoring is correct and preserves the expected behavior, I’ll unit test the code along the way using ScalaTest. ScalaCheck, JUnit, or TestNG would work just as well, but I used ScalaTest.

First Things First

“What, exactly,” some of you may be asking, “is Levenshtein string distance? Some kind of Teutonic tailoring terminology?” Not at all. It’s a way of measuring how alike or different two strings of symbols are. For example, the string “sturgeon” is a lot more similar to “surgeon” than to “urgently”. “Sturgeon” and “surgeon” are only a single letter (t) apart. They have a string distance of 1. “Sturgeon” and “urgently” share some letters, but each has some letters not present in the other. So what’s their string distance? It’s not so obvious now.

String distance is useful. The use that most quickly springs to mind is spelling correction. If I type “computwr” a string distance algoritm could tell us that “computwr” is very close to the dictionary word “computer”. But there’s more to it than that. There are a lot of fuzzy problems in which you want to find which two sets of complex data are most similar. One way to solve a problem like that is to encode it into a string compare using string distance. For example, you could write a program to find pen strokes in an image and encode their shapes (up, curve left, down, angle right, etc) as a string which could be compared to known encodings for handwritten letters. By finding the closest matches you can create a handwriting recognition program. String distance is useful in DNA analysis, of course, recognizing patterns in signals, and a host of other situations.

The Levenshtein string distance algorithm was developed by Vladimir Levenshtein in 1965. It can easily tell us the distance from “sturgeon” to “urgently”. This algorithm breaks down string transformation into three basic operations: adding a character, deleting a character, and replacing a character. Each of these operations is assigned a cost, usually a cost of 1 for each operation. Leaving a character unchanged has a cost of 0. So to go from “sturgeon” to “surgeon” you leave the “s” unchanged for a cost of 0. Then you add a “t” for a cost of 1. All the other letters also remained unchanged, so the total cost is 1, just as we expected.

To change “sturgeon” to “urgently” is harder. They have the same number of letters, so we could just do a replacement on each one for a distance of 8. But is that the shortest distance? What if we try to re-use that “urge” from “sturgeon”? Can we re-use the “n”? Does that help? What about the “t”? We need an algorithm that we can follow.

The Grid

What we need is a way to find the cheapest combination of operations which changes the first string to the second. That’s the Levenshtein algorithm. It works like this. Write the first string vertically from top to bottom. To the right of each letter write 1, 2, 3, etc. Write a 0 above the number 1. Then write the second string horizontally and again add the numbers 1, 2, 3 etc. to the right of the 0. It will look something like this:

    u r g e n t l y
  0 1 2 3 4 5 6 7 8
s 1
t 2
u 3
r 4
g 5
e 6
o 7
n 8

That’s the first step. The grid remains un-filled-in at this point. Can you guess what the grid positions are going to hold? They are going to hold the cost to convert the various prefixes of “sturgeon” to the various prefixes of “urgently”. The position at the intersection of row r4, and the column a2, for example, will contain the cost to convert “stur” to “ur”. We fill in these positions (left-to-right then top to bottom) with the smallest of the following three numbers:

• The number above the current position plus one.
• The number to the left of the current position plus one.
• The above-left number if row letter and column letter are the same, or the above-left number plus one otherwise.

When we finish, the bottom right corner contains the cost to convert the first string to the second.

But Why? (Understanding the Algorithm)

Those are some shockingly simple rules! Let’s examine how they translate into string distance.

First, what are those numbers 0 to n that we write along the top and left? They’re not just indices. The numbers along the top represent the cost to convert the empty string to the various prefixes of “urgently”. The cost is 0 to convert “” to “”, 1 to convert “” to “u”, 2 to convert “” to “ur”, and so on. The numbers on the left are the cost to convert the prefixes of “sturgeon” to the the empty string . “” to “” is 0, “s” to “” is 1, and so forth. These costs are obvious. The only way to convert “” to “urgently” is to add eight letters. There’s nothing to delete, nothing to replace. The only way to convert “sturgeon” to “” is to delete all eight letters.

Each position, as we have established, represents the cost to convert the string of characters down the left side ending in the current row’s character into the the string of characters along the top ending at the current column’s character. Put another way, for any given position let’s call the current row’s letter A, and the current column’s letter B. If we use a colon (:) to indicate string concatenation then the beginning string, the one along the left of the grid, can be written Prefix1:A. So for our example word “sturgeon” if we look at a position in row e6 then Prefix1 is “sturg” and the final letter, which we’re calling A, is “e”.

So, speaking in terms of Prefix1, Prefix2, A, and B, we use the following inputs:

• The cost to change Prefix1 to Prefix2:B (the number above the current position).
• The cost to change Prefix1:A to Prefix2 (the number to the left).
• The cost to change Prefix1 to Prefix2 (the above-left number).

If we know the costs of these three conversions we can find the cost to change Prefix1:A to Prefix2:B using this logic:

• We know that if converting Prefix1 to Prefix2:B has a cost of X, then Prefix1:A can be converted to Prefix2:B for the cost of X plus the cost of deleting A, or X + 1.
• We know that if converting Prefix1:A to Prefix2 has a cost of Y, then Prefix1:A can be converted to Prefix2:B for the cost of Y plus the cost of adding B, or Y + 1.
• We know that if converting Prefix1 to Prefix2 has a cost of Z, then Prefix1:A can be converted to Prefix2:B for the cost of Z plus the cost of replacing A with B, or Z + (0 or 1 depending on whether A = B).

If you understand the above logic, then you understand this really neat algorithm. It’s a quintessential example of dynamic programming. The solution is built up by solving simpler problems. You start with the trivial case of converting to and from the empty string, and then you build up the solution for the prefixes until you have the complete solution.

In the grid’s initial configuration there is only one location for which we know all three costs and that is row s1, column u1. That’s the only space for which all three neighboring values (above, left, and above-left) are populated. After we fill in this one, there are two more spaces available to us. Those are (t2, u1) and (s1, r2). Ordinarily, though, the spaces are populated line by line.

A Simple Example

Let’s do a quick example by hand. Then we’ll take a stab at implementing it in code. What’s the string distance from “hat” to “tape”? First, our empty grid:

    t a p e
  0 1 2 3 4
h 1
a 2
t 3

The first space is row h, column t. The letters are different so our choices are 1 + 1 (above), 1 + 1 (left), or 0 + 1 (above-left). 0 + 1 is the smallest value, so the first space gets populated with 1. The next space has choices 2 + 1 (above), 1 + 1 (left), or 1 + 1 (above-left). 1 + 1 is the lowest, so we fill in the second space with 2. Once we finish the row, we have this:

    t a p e
  0 1 2 3 4
h 1 1 2 3 4
a 2
t 3

The next space is row a, column t. Our choices are 1 + 1 (above), 2 + 1 (left), or 1 + 1 (above-left). 1 + 1 is the smallest value, so the space gets populated with 2. The next space has choices 2 + 1 (above), 2 + 1 (left), or 1 + 0 (above-left). Why 1 + 0? Because the above-left value is 1 and both letters for this space are “a” so we can replace “a” with “a” for free. Go ahead and finish the whole grid. This is the result:

    t a p e
  0 1 2 3 4
h 1 1 2 3 4
a 2 2 1 2 3
t 3 2 2 2 3

The strings “hat” and “tape” have a distance of 3.

Some Code, Finally

As fascinating as Levenshtein distance is, and as much more as there is to say on the topic, the time has come to write some code. Here’s a Scala implementation that is very close to the pencil-and-paper approach that we just performed.

import scala.Math.min
import scala.Math.max

object StringDistance {
  def stringDistance(s1: String, s2: String): Int = {
    def minimum(i1: Int, i2: Int, i3: Int) = min(min(i1, i2), i3)

    val dist = new Array[Array[Int]](s1.length + 1, s2.length + 1)

    for (idx <- 0 to s1.length) dist(idx)(0) = idx
    for (jdx <- 0 to s2.length) dist(0)(jdx) = jdx

    for (idx <- 1 to s1.length; jdx <- 1 to s2.length)
      dist(idx)(jdx) = minimum (
        dist(idx-1)(jdx  ) + 1,
        dist(idx  )(jdx-1) + 1,
        dist(idx-1)(jdx-1) + (if (s1(idx-1) == s2(jdx-1)) 0 else 1)
      )

    dist(s1.length)(s2.length)
  }

Do you see what I mean when I say it’s close to the pencil-and-paper approach? We actually construct a two-dimensional Array to represent the grid we drew earlier. It’s a very literal implementation.

To explain the code briefly, we declare a singleton StringDistance having a single method called stringDistance. Within this method we declare a 3-argument minimum method. (I wonder why there’s no “def min(params: Int*): Int” defined in scala.Math.) Then we create an array called “dist” and populate the top row and leftmost column in lines 8-11. The for loop on line 13 cycles through each array position from left to right then top to bottom, and populates them according to the Levenshtein logic. Finally, once the grid is filled in we return the number in the bottom right position.

The job’s not done yet, of course. We haven’t written our unit tests. Here’s the test I wrote:

import org.scalatest._

class StringDistanceSuite extends FunSuite with PrivateMethodTester {

  test("stringDistance should work on empty strings") {
    assert( StringDistance.stringDistance(   "",    "") == 0 )
    assert( StringDistance.stringDistance(  "a",    "") == 1 )
    assert( StringDistance.stringDistance(   "",   "a") == 1 )
    assert( StringDistance.stringDistance("abc",    "") == 3 )
    assert( StringDistance.stringDistance(   "", "abc") == 3 )
  }

  test("stringDistance should work equal strings") {
    assert( StringDistance.stringDistance(   "",    "") == 0 )
    assert( StringDistance.stringDistance(  "a",   "a") == 0 )
    assert( StringDistance.stringDistance("abc", "abc") == 0 )
  }

  test("stringDistance should work where only inserts are needed") {
    assert( StringDistance.stringDistance(   "",   "a") == 1 )
    assert( StringDistance.stringDistance(  "a",  "ab") == 1 )
    assert( StringDistance.stringDistance(  "b",  "ab") == 1 )
    assert( StringDistance.stringDistance( "ac", "abc") == 1 )
    assert( StringDistance.stringDistance("abcdefg", "xabxcdxxefxgx") == 6 )
  }

  test("stringDistance should work where only deletes are needed") {
    assert( StringDistance.stringDistance(  "a",    "") == 1 )
    assert( StringDistance.stringDistance( "ab",   "a") == 1 )
    assert( StringDistance.stringDistance( "ab",   "b") == 1 )
    assert( StringDistance.stringDistance("abc",  "ac") == 1 )
    assert( StringDistance.stringDistance("xabxcdxxefxgx", "abcdefg") == 6 )
  }

  test("stringDistance should work where only substitutions are needed") {
    assert( StringDistance.stringDistance(  "a",   "b") == 1 )
    assert( StringDistance.stringDistance( "ab",  "ac") == 1 )
    assert( StringDistance.stringDistance( "ac",  "bc") == 1 )
    assert( StringDistance.stringDistance("abc", "axc") == 1 )
    assert( StringDistance.stringDistance("xabxcdxxefxgx", "1ab2cd34ef5g6") == 6 )
  }

  test("stringDistance should work many operations are needed") {
    assert( StringDistance.stringDistance("example", "samples") == 3 )
    assert( StringDistance.stringDistance("sturgeon", "urgently") == 6 )
    assert( StringDistance.stringDistance("levenshtein", "frankenstein") == 6 )
    assert( StringDistance.stringDistance("distance", "difference") == 5 )
    assert( StringDistance.stringDistance("java was neat", "scala is great") == 7 )
  }

}

It tests several special cases as well as the general case. All we have to do is compile our StringDistance object and the StringDistanceSuite unit test, fire up the scala interpreter and run the test:

scala> (new StringDistanceSuite).execute()
Test Starting - StringDistanceSuite: recursiveStringDistance should work on empty strings
Test Succeeded - StringDistanceSuite: recursiveStringDistance should work on empty strings
Test Starting - StringDistanceSuite: recursiveStringDistance should work on equal strings
Test Succeeded - StringDistanceSuite: recursiveStringDistance should work on equal strings
Test Starting - StringDistanceSuite: recursiveStringDistance should work where only inserts are needed
Test Succeeded - StringDistanceSuite: recursiveStringDistance should work where only inserts are needed
Test Starting - StringDistanceSuite: recursiveStringDistance should work where only deletes are needed
Test Succeeded - StringDistanceSuite: recursiveStringDistance should work where only deletes are needed
Test Starting - StringDistanceSuite: recursiveStringDistance should work where only substitutions are needed
Test Succeeded - StringDistanceSuite: recursiveStringDistance should work where only substitutions are needed
Test Starting - StringDistanceSuite: stringDistance should work many operations are needed
Test Succeeded - StringDistanceSuite: stringDistance should work many operations are needed

scala>

Refactoring: Reduced Memory Usage

The code passes all the tests, so let’s take things a step further. One shortcoming of our implementation is that it can require a lot of memory. That array has to be of size (n+1)*(m+1) where n and m are the lengths of the two strings we’re comparing. If you want to apply the method to strings that are a few hundred characters long (or longer) then you’re starting to talk about some serious memory requirements. How can we reduce the amount of memory required? Can you think of a way?

Once we complete one row of the grid, we use it again as an input when we compute the next row. But after that we’re done with it. Why leave it to clutter the heap? Let’s tweak our implementation slightly. Try rewriting the method using only two rows. Fill the first row with the initial 0, 1, 2, etc. Then use one to compute the other over and over. Think about how you would implement this, then have a look at my solution below.

  def stringDistance(s1: String, s2: String): Int = {
    def minimum(i1: Int, i2: Int, i3: Int) = min(min(i1, i2), i3)

    var dist = ( new Array[Int](s1.length + 1),
                 new Array[Int](s1.length + 1) )

    for (idx <- 0 to s1.length) dist._2(idx) = idx

    for (jdx <- 1 to s2.length) {
      val (newDist, oldDist) = dist
      newDist(0) = jdx
      for (idx <- 1 to s1.length) {
        newDist(idx) = minimum (
          oldDist(idx) + 1,
          newDist(idx-1) + 1,
          oldDist(idx-1) + (if (s1(idx-1) == s2(jdx-1)) 0 else 1)
        )
      }
      dist = dist.swap
    }

    dist._2(s1.length)
  }

This one uses a Pair (also called a Tuple2) containing two one-dimensional arrays, instead of a 2*(n+1) array. Pair happens to have the very handy “swap” method which we can use to trade out the rows when we’ve finished one and are ready to compute the next.

This is where our unit tests really show their worth. No need to wonder whether this new code really does work. We just recompile, run the tests again, and we can see that the code still gives us the expected results.

Refactoring: Recursion, Kind Of

What are some other ways we could write this code? Can it be improved? I thought I would try to replace the iteration in the previous implementations with recursion. Here’s what that code looks like:

  def stringDistance(s1: String, s2: String): Int = {
    def newCost(ins: Int, del: Int, subst: Int, c1: Char, c2:Char) =
      Math.min( Math.min( ins+1, del+1 ), subst + (if (c1 == c2) 0 else 1) )

    def getNewCosts(s1: List[Char], c2: Char, delVal: Int, prev: List[Int] ): List[Int] = (s1, prev) match {
      case (c1 :: _ , substVal :: insVal :: _) =>
        delVal :: getNewCosts(s1.tail, c2, newCost(insVal, delVal, substVal, c1, c2), prev.tail)
      case _ => List(delVal)
    }

    def sd(s1: List[Char], s2: List[Char], prev: List[Int]): Int = s2 match {
      case Nil => prev.last
      case _ => sd( s1, s2.tail, getNewCosts(s1, s2.head, prev.head+1, prev) )
    }

    (s1, s2) match {
      case (`s2`, `s1`) => 0
      case (_, "") | ("", _) => max(s1.length, s2.length)
      case _ => sd(s1.toList, s2.toList, (0 to s1.length).toList)
    }
  }

It’s a pretty naïve implementation, actually. It just replaces the the repetition of the two for loops with the repetition of the recursion of the two methods sd and getNewCosts. The sd method is even tail-recursive, allowing scala to optimize it. It does the same basic thing as the for loop version, though. It recurses through the characters of a row in the getNewCosts method, and it recurses through the rows of the grid in the sd method.

It looks more complicated than the previous implementations. It’s harder to read. But it passes our unit tests, so we can be pretty sure it’s correct.

Refactoring: List Methods

After the last implementation, I thought it looked a little sloppy. I wondered whether I could improve the situation by using some of the many useful methods built into scala’s List class. Here is the comparatively brief code that resulted:

  def stringDistance(s1: String, s2: String): Int = {
    def sd(s1: List[Char], s2: List[Char], costs: List[Int]): Int = s2 match {
      case Nil => costs.last
      case c2 :: tail => sd( s1, tail,
          (List(costs.head+1) /: costs.zip(costs.tail).zip(s1))((a,b) => b match {
            case ((rep,ins), chr) => Math.min( Math.min( ins+1, a.head+1 ), rep + (if (chr==c2) 0 else 1) ) :: a
          }).reverse
        )
    }
    sd(s1.toList, s2.toList, (0 to s1.length).toList)
  }

Like I say, it’s brief. Those List methods give you a lot of mileage.

Refactoring: Real Recursion

The more I looked at my previous attempt at a recursive solution, the more I realized how hare-brained it was. It wasn’t real recursion. It was just iteration using the stack. So I went back to the drawing board. If I want to know the final answer, the value in the bottom right position, how do I get it? I apply my three rules to the above, left, and above-left positions. How do I get those positions? Apply the rules again. That’s real recursion. Here’s my first stab at it:

  def stringDistance(s1: String, s2: String): Int = {
    def min(a:Int, b:Int, c:Int) = Math.min( Math.min( a, b ), c)
    def sd(s1: List[Char], s2: List[Char]): Int = (s1, s2) match {
      case (_, Nil) => s1.length
      case (Nil, _) => s2.length
      case (c1::t1, c2::t2)  => min( sd(t1,s2) + 1, sd(s1,t2) + 1,
                                     sd(t1,t2) + (if (c1==c2) 0 else 1) )
    }
    sd( s1.toList, s2.toList )
  }

Now, THAT is a nice looking recursive function. That’s more like it. See the pattern match block? If we try to convert from any string to the empty string or from the empty to a non-empty string then we just use the string length. You see? That takes care of the positions along the top and left of the grid. All the others are determined in the last case. That last case just applies our three rules. That is so short and simple. It’s a thing of beauty.

The only problem? It doesn’t work.

It’s technically correct. It will return correct answers … eventually. Or, rather, I think it will. I can’t be sure because it’s too slow to complete my unit tests! For anything but very short inputs (4 or 5 characters), the function takes a long time to return. Why? Let’s look at how many recursive calls are made for some inputs.

If we pass use strings “a” and “b” we pass by the “(_, Nil)” and “(Nil, _)” cases in the first call to function sd, because both our strings (Lists, actually) are non-empty. This results in three more calls to sd. Each of these three calls includes an empty List of characters, so there is no more recursion. That’s a total of four calls to sd for strings “a” and “b”.

What about “ab” and “xy”? Think about it for a moment? Step through the function in your head. How many calls to sd will there be for “ab” and “xy”?

Have you done it? I count 19. What about “abc” and “xyz”? I’ll save you the trouble. It’s 94. For length 4 strings it’s 481. For length 5 it’s 2524. Length 6 is 13,483. Then 73 thousand, then 400 thousand, then 2 million and so forth. Why so many calls? Each position in the grid is computed using all the positions to the left and all the positions above the current position. So a position in the top left will be computed and recomputed many times.

There is a way to get around this, of course. You probably already have some ideas. We’re going to do something called memoization. When you memoize a function, you make it remember results that it computed previously without having to actually recompute them. I’ll do that by caching results in a map. The map’s key is a Pair of List[Char]s, the inputs to my inner function sd, and its data is an Int, the return type of sd. I will modify sd to first check the map to see if the result for the current parameters has already been cached. If so, we simply return it. If not, we compute the value, cache it, and return it.

  def stringDistance(s1: String, s2: String): Int = {
    val memo = scala.collection.mutable.Map[(List[Char],List[Char]),Int]()
    def min(a:Int, b:Int, c:Int) = Math.min( Math.min( a, b ), c)
    def sd(s1: List[Char], s2: List[Char]): Int = {
      if (memo.contains((s1,s2)) == false)
        memo((s1,s2)) = (s1, s2) match {
          case (_, Nil) => s1.length
          case (Nil, _) => s2.length
          case (c1::t1, c2::t2)  => min( sd(t1,s2) + 1, sd(s1,t2) + 1,
                                         sd(t1,t2) + (if (c1==c2) 0 else 1) )
        }
      memo((s1,s2))
    }

    sd( s1.toList, s2.toList )
  }

There. That uglies up my function somewhat, but at least it’s usable now. And it passes my unit tests, so I’m reasonably assured that it’s right.

Memoization only works if you expect the same result for each identical function call. If your function takes input from stdin, for example, you can’t memoize that. Or if it has a random component. Or if, for any other reason, its return value is not always the same for the same inputs. You can memoize functions in different ways. There’s a post on Michid’s Weblog about a more general solution, a memoizing class which wraps existing functions to give you a memoized version.

PHEW!

Ok, I’ve tried to keep my rambling to a minimum in this post but it’s still a doozy. The things I wanted to get across are:

• The usefulness of Levenshtein distance in solving a variety of problems.
• How to understand the Levenshtein distance algorithm and why it works.
• How to use unit tests to improve your code while maintaining some assurance that the new code still has the correct behavior.
• Some of the different ways of implementing Levenshtein.

In the end, I think I like the second implementation (the one that switches out the two rows) and the last implementation the best. The second one seems to have good performance. I did some informal performance tests and it has a good mix of performance and simplicity. The last one, the memoized recursive one, appeals to me because it is in a more functional style and still has respectable performance.

Don’t forget to subscribe to my RSS feed, or follow this blog on Twitter.
Copyright © 2009 Matthew Jason Malone

js_runit is now a standalone JavaScript testing framework. So if your into a minimalist JavaScript testing experience, this tool is for you. Two files to download. The html file and the JavaScript file. Included with this is an example of Passive View with unit tests.

To really appreciate this see the passive view example.

The unit tests for the above example can be found here.

Happy unit testing.

Quick Hack:

Right-click the unit test project in VS.NET -> Unload

Right-click the unit test project in VS.NET -> Edit [project file]

Navigate to <ProjectTypeGuids>…

Remove “{3AC096D0-A1C2-E12C-1390-A8335801FDAB};” from the listing.

Save the file. Reopen in VS.NET.

Finito!!!

Don’t forget to add this line back to have the project behave like a VS.NET Unit Test project after adding the Adapter reference!!!

I am trying, with only partial success, to apply what I’ve learned in Working Effectively With Legacy Code by Michael C. Feathers.

Feathers is a huge advocate of test-driven development. He puts it out there on page xvi: “Code without tests is bad code.” He defines “legacy code” as, strictly speaking, any code that isn’t already under unit tests. At first it struck me as a funny definition, because obviously lots of code is written today–even by me–without unit tests, and how can it be right to refer to software nobody’s even thought of yet as “legacy”? But for purposes of the book it works.

The bulk of this text consists of techniques for making changes in existing code, either while bringing into a unit testing regime or while carefully working around the lack of one. Thoughtfully, each of the chapters is titled with a question or complaint, to wit:

• How Do I Add a Feature?
• I Need to Make Change, but I Don’t Know What Tests to Write
• My Test Code Is in the Way

and so on.

Within each chapter, Feathers gives one or more coping techniques. The effect is quite a lot like that of Design Patterns by Gamma et al., sort of an indexed glossary of solutions. With names. Throughout the text, solutions are referenced by name and page number, such as when the Sprout Method (59) solution suggests use of Pass Null (111).

Sprout Farming

And that’s the one I want to talk about today: the Sprout Method solution. I’ve used this a lot recently and it makes sense even if it looks weird at first.

Here’s the problem.

You need to add some code to a method, you might need to do it in multiple places in the system, you want the changes to be unit-tested, and you need to do it right away without a lot of refactoring.

Here’s the solution.

You write the added code as a new method and call it where needed, rather than typing the new code in every place it goes.

This can be kind of ugly at first, because your original method might have thirty lines of logic-logic-logic and then there’s this one crazy method call in the middle of it that does about five lines of stuff and it’s not obvious why you didn’t put it in the main method.

But here’s the thing. The new method can be put under unit test, and then you’ve gained a little bit more control over the next change and the change after that. Maybe bits and pieces of the big methods migrate into your little new method over time. Maybe your big methods eventually turn into lots of these little methods, and then you realize the big methods are now completely unit tested anyway and you’ve succeeded!

What it’s good for

If today were endless, we’d spend it refactoring everything into logical units and placing it all under test with JUnit or NUnit or the like. That would be cool. But alas, the world is not that cool, and we don’t get to muck around with code that already works just to make it hypothetically easier to change in the future. What we can do is get the most benefit from the changes that we have to make anyway.

So Sprout Method (59) is a good start. Long story short, when you need to add a few (or more) lines of code to an existing method, consider making the new code a testable method of its own.

“The Art of Unit Testing” von Roy Osherove ist im Mai 2009 bei Manning erschienen. Das Buch behandelt Unit Testing in C#.

 
Zum Inhalt
Kapitel 1 zeigt die Basis von Unit Tests. Diese umfasst eine Definition der Begriffe, eine kurze Einführung zu Test-Driven Development (TDD) und eine Abgrenzung gegen Integrationstests. Den nicht alles, was sicht Unit Test nennt, ist auch wirklich einer.

Kapitel 2 hilft einem durch den ersten Unit Test. Nach der Auflistung der Vorteile die ein entsprechendes Test-Framework bringen folgt ein Rundgang durch NUnit. Dieser reicht von der Installation über die Einrichtung eines Test-Projekts zu Namenskonventionen bis hin zur Ausführung der Tests.

Wie man mit Stubs die Abhängigkeiten aufbrechen kann wird in Kapitel 3 gezeigt. Das Beispielprojekt hat zu beginn eine direkte Abhängigkeit zum Dateisystem. In 2 Schritten wird der Code so umgebaut, dass die Abhängigkeit aufgelöst werden kann. Mit einer kleinen Hilfsklasse, dem Stub, kann man dann genau die Antwort zurück liefern, die man für den Test der Übergeordneten Klasse benötigt. Dies kann in dem Beispiel sowohl ein Boolean sein wie auch eine Exception des Dateisystems.
Es folgt ein Abstecher ins Thema Dependency Injection. Denn der Stub, aber auch die konkrete Klasse mit der Logik, muss ja mit der aufrufenden Klasse verbunden werden. Es werden die Vor- und Nachteile der jeweiligen Ansätze dargelegt, damit man selber das zu seinem Problem passende auswählen kann.

Kapitel 4 zeigt wie man mit Mock Objekten die Interaktion mit der zu testenden Klasse prüfen kann. Mit einem selbst geschriebenen Mock wird das Prinzip erklärt, bevor in Kapitel 5 das Mock-Framework Rhino Mocks erklärt wird. Eine Übersicht der anderen verbreiteten Frameworks (wie TypeMock, NMock und NUnit.Mocks) rundet das Kapitel ab.

Die Organisation von Tests, Test Patterns und Continuous Integration bilden das Kapitel 6, während Kapitel 7 die Qualität der Tests behandelt. Lesbarkeit und Verständlichkeit sind Dinge, die auch bei Tests bedacht werden müssen.

Kapitel 8 hilft bei der Integration von Unit Tests in die Firma. Hierbei geht es nicht um die technische, sondern um die organisatorische Ebene. Osherove berichtet über seine Erfahrungen und liefert Antworten zu häufig aufkommenden Fragen und Problemen.

Wie man mit Legacy Code arbeiten kann zeigt Kapitel 9. Grosse Projekte ohne wirkliche Unit Tests benötigen spezifische Vorgehensweise. Auch hier werden Vor- und Nachteile der Ansätze aufgezeigt. Der Einsatz von Tools kann einem auch hier das Leben erleichtern. Depender oder NDepend können Abhängigkeiten anzuzeigen, was als Ausgangslage für Unit Tests dienen kann. Für Java Entwickler werden die Tools JMockit (vergleichbar mit TypeMock) und Vise erwähnt.

Design und Testbarkeit werden noch einmal ausführlich im Anhang A behandelt. Anhang B bietet eine kompakte Auflistung von Tools und Frameworks aus allen Bereichen von Tests.

 
Fazit
Ein sehr gutes Buch zur Kunst des Unit Testings. Auf rund 300 Seiten erfährt man unzählige Möglichkeiten zum sauberen testen. Das Buch richtet sich eher an Leser mit Erfahrungen im Bereich Unit Testing. Für Anfänger mag sich nicht immer erschliessen, worin die Vorteile der jeweiligen Ansätze liegen. Hat man sich aber mit den Problemen selber herumgeschlagen, braucht es nicht mehr viele Worte. Ich fand das Buch eine ideale Ergänzung zu Test Driven.
Der Anhang A hätte ein eigenes, „richtiges“ Kapitel verdient und einige Grafiken hätte man nochmals überarbeiten können. Davon abgesehen ist das Buch sehr gut.

 
Zum Buch
The Art of Unit Testing with Examples in .NET von Roy Osherove, 2009 Manning, ISBN 978-1-933988-27-6, 320 Seiten

Los miembros y métodos en C#, pueden gozar de varios modificadores de acceso. Amigos míos, no solo de públicos y privados vive el hombre.

Hoy vamos a hablar de uno de ellos.

El modificador de acceso internal.

El modificador de acceso internal, es aquel que nos va a permitir acceder a la clase, miembro, método etc. desde el propio ensamblado.

Que quiere decir esto, pues implica que dentro de un mismo ensamblado vamos a poder acceder a una clase internal como si de una clase pública se tratara, pero fuera de este ensamblado esta clase no va a ser accesible ya que no la estamos exponiendo.

Trabajar con clases internal, considero que es una buena práctica, ya que en los distintos assemblies que vamos generando, únicamente vamos a exponer lo que consideramos fundamental, de manera que el susodicho assembly, queda mucho más encapsulado. Además si el software que estamos desarrollando va a ser distribuido para que lo usen terceros, todavía tenemos más razones para usar este tipo de modificador de acceso.

Los métodos internal y las pruebas unitarias (unit test) o como acceder a aquello que no expones.

Hasta aquí, me encanta el modificador de acceso internal, pero ¿qué ocurre cuando tenemos que crear las pruebas unitarias para métodos internal?

Pues muy sencillo, que como no estamos exponiendo muchas clases y métodos, resulta que no lo podemos probar.

Jajajajaja, bueno, esto no es así exactamente aunque sí que es verdad que es una de las primeras cositas que encuentras (si tu desarrollo es TDD compliance claro).

La realidad es que no estamos exponiendo muchas clases por lo tanto no deberíamos poder probarlas, pero el Framework de ASP.NET precisamente cuenta “un mecanismo” para realizar esto.

El atributo InternalsVisibleTo.

Pues si amigos míos, el Framework de ASP.NET cuenta con atributo llamado InternalsVisibleTo que nos va a permitir acceder a clases internal desde assemblies externos.

El código necesario sería el siguiente:
[assembly: InternalsVisibleTo("MiAssemblyName.Test")]

Y esto, tenemos que colocarlo en el AssemblyInfo de nuestro proyecto.

Nada más por hoy mis druguitos. Aquí se acaba esta sesión de ultraviolencia. Os dejo con The XX, que además de ser unos chavales que hacen una música que me recuerda a los títulos de crédito de documentos tv, son geniales.

NCover is an excellent tool that allows you to calculate how much of your code is being hit by your unit tests. Basically what it does is a profile of your application. So you can run your tests through any application and NCover will tell you the coverage of your tests. NCover is not an open source free tool, however it does have a Community Edition that will do the work for this tutorial which you can download from here after going through the registration.

Since I have already showed how to run NUnit tests from the console, we are going to calculate the  coverage using NCover with NUnit. And taking advantage of the folder structure used we will continue with the following structure:

ncover folder

In order to make this available to the whole team, we are going to create a batch file (.bat) that will run the command for us (this will be usefull when integrating it with Nant). As I explained earlier, ncover has to execute an application in order to do the profiling. In our case, we want to run the NUnit tests, so this application will be nunit-console that rests in \lib\nunit 2.4.8 folder. We covered the parameters that NUnit needs in order to run our tests in my previous post, so the only thing left to do is run ncover. The batch file will in the trunk folder, and will contain the following command:

.\NCover\NCover.console.exe  .\nunit\nunit-console.exe /xml=.\build-result\tests.xml .\Concur.Lib.Tests\bin\debug\Concur.Lib.Tests.dll

We can optionally add a parameter in order to store the coverage output file into a different folder, like this: //x .\build-result\coverage.xml, so the batch file would look like this:

.\NCover\NCover.console.exe //x .\build-result\coverage.xml .\nunit\nunit-console.exe /xml=.\build-result\tests.xml .\Concur.Lib.Tests\bin\debug\Concur.Lib.Tests.dll

And finally you can use your favorite xsl in order to visualize the results. There’s a very cool xsl file in the NCover folder called “coverage.xsl” which you can use by editing the xml to add the reference like this:
<?xml version=”1.0″ encoding=”utf-8″ standalone=”no”?>
<?xml-stylesheet type=”text/xsl” href=”coverage.xsl”?>

Adding this file to the build-result folder is a good idea too.

So we have successfully calculated our NCoverage coverage for our project, and left a batch file available to the whole team to run it in their local machine. Simple as it is, this leaves us one step closer to having a pretty decent build mechanism.

Test Next Generation !! Definitely TestNG.

TestNG is a testing framework designed to simply a broad range of testing needs, from unit testing (testing a class in isolation of the others) to integration testing (testing entire systems made of several classes, several packages and even several external frameworks, such as application servers).

TestNG is a testing framework inspired from JUnit and NUnit but introducing some new functionalities that make it more powerful and easier to use, such as:

• JDK 5 Annotations (JDK 1.4 is also supported with JavaDoc annotations).
• Flexible test configuration.
• Support for data-driven testing (with @DataProvider).
• Support for parameters.
• Allows distribution of tests on slave machines.
• Powerful execution model (no more TestSuite).
• Supported by a variety of tools and plug-ins (Eclipse, IDEA, Maven, etc…).
• Embeds BeanShell for further flexibility.
• Default JDK functions for runtime and logging (no dependencies).
• Dependent methods for application server testing.

TestNG is designed to cover all categories of tests:  unit, functional, end-to-end, integration, etc…

Writing a test is typically a three-step process:

1. Write the business logic of your test and insert TestNG annotations in your code.
2. Add the information about your test (e.g. the class name, the groups you wish to run, etc…) in a testng.xml file or in build.xml
3. Run TestNG

Requirements:

TestNG runs on JDK 1.4 and 5.  The examples shown in this documentation assume JDK 5 and therefore, use JDK 5 annotations, but they can easily be translated to JDK 1.4 JavaDoc-type annotations.

Integration with other tools:

TestNG can be integrated with Eclipse, IDEA IntelliJ and NetBeans with plugins.
Also by ant & maven you can integrate this TestNG to any Java Project.

Simple Example:

I am giving simple example from testng.org site to visualize how easy to configure TestNG to in any project.

package example1;

import org.testng.annotations.*;

public class SimpleTest {

@BeforeClass
public void setUp() {
// code that will be invoked when this test is instantiated
}

@Test(groups = { “fast” })
public void aFastTest() {
System.out.println(“Fast test”);
}

@Test(groups = { “slow” })
public void aSlowTest() {
System.out.println(“Slow test”);
}

}

The method setUp() will be invoked after the test class has been built and before any test method is run.  In this example, we will be running the group fast, so aFastTest() will be invoked while aSlowTest() will be skipped.

Things to note:

• No need to extend a class or implement an interface.
• Even though the example above uses the JUnit conventions, our methods can be called any name you like, it’s the annotations that tell TestNG what they are.
• A test method can belong to one or several groups. These groups will be used at runtime to determine what test methods should be invoked..

Once you have compiled your test class into the build directory, you can invoke your test with the command line, an ant task (shown below) or an XML file:

<project default=”test”>

<path id=”cp”>
<pathelement location=”lib/testng-testng-4.4-jdk15.jar”/>
<pathelement location=”build”/>
</path>

<taskdef name=”testng” classpathref=”cp”
classname=”org.testng.TestNGAntTask” />

<target name=”test”>
<testng classpathref=”cp” groups=”fast”>
<classfileset dir=”build” includes=”example1/*.class”/>
</testng>
</target>

</project>

Use ant to invoke it:

D:> ant
Buildfile: build.xml

test:
[testng] Fast test
[testng] ===============================================
[testng] Suite for Command line test
[testng] Total tests run: 1, Failures: 0, Skips: 0
[testng] ===============================================

BUILD SUCCESSFUL
Total time: 4 seconds

You can view the results from browser for above file test-output\index.html

Why TestNG:

• Parametric Testing Ability: One of my favorite features of TestNG is its parametric testing ability, which allows you to create generic test cases and then vary the test values– you can use parameters from XML files or even use the Data Provider feature for a richer parameter type. In fact, for awhile, because it’s my bag baby, I’ve been espousing TestNG’s out of the box parametric testing features as a reason to use TestNG as an opposed to JUnit for higher level testing. JUnit 4, however, now supports parametric tests– and you’ll find that its parametric testing is rather similar to TestNG’s Data Provider.
• Configurable Test Suite: Second one should not dirty his/her hands to create test suite. Here you just write xml & you done with your test suite. You can define which test should executed etc.
• Logical Groups: Other feature is grouping. You can create logical groups which may be class level or method level or both by simply annotations. Suppose you have different packages in BO layer so you can define class level group for Business Layer for all classes. & you can run TestNG only for those groups.
• Reports: You can generate different reports from email-able reports to deep level reports. You also can generate diff. reports using custom style sheets.

Reports:

There are different types of Report generated in one go:

• Chronological Reports: Methods run, sorted chronologically
• Alphabetical Reports: Methods run, sorted alphabetically
• Groups Report: Groups used for this test run
• Not Run Methods: Methods that were not run
• Reporter Out: If you used Reporter API provided by TestNG.
• Emailable-report: Small & Sweet Report may be useful for client or PM.

If possible just try out in your project. It really worth to try.

Looking for more info about running unit tests in Resharper, I found this post: MSTest ReSharper 4 Plugin.

Resharper 4.5 support MSTest natively (here), so it’s obsolete but not useless.

During my early tests, I found Resharper running slowly my unit tests, but more customizable.

Technorati tags: Visual Studio, Resharper, Unit Test

Olá a todos! Hoje vamos falar de mais um capítulo da série “Problema e solução”.

Problema:

Ao configurar uma Team Build no VSTS de um projeto que compila e que roda os testes unitários tudo certinho, a build roda, porém o status dela fica como “Partially Succeeded” porque os testes unitários falharam e, ao analisar o log, você encontra a seguinte linha no final:

MSBUILD : warning MSB6006: “MSTest.exe” exited with code 1.

O mais interessante é que se você roda os testes unitários a partir do Test List da Solution, todos funcionam, não é?

Causa:

Pois bem. Por algum motivo, o erro se dá por conta do usuário TFSSERVICE* não ter permissão na pasta onde o resultado do build foi colocado (ou “dropado”).

Solução:

Basta permitir escrita (tanto NTFS quando de compartilhamento) para pra o usuário TFSSERVICE na pasta de build drop.

*Este nome pode variar de acordo com a instalação realizada.

[]’s

Ricardo Serradas

Javascript is a language used in web development. I would argue that it is Object Oriented (OO). And due to this the principles guiding development in an OO language apply.

When developing software, at its basic level, you have three responsibilities that need to be managed.

1. Input: consist of user actions.
2. Output: change to the web browser.
3. Logic: any piece of code where a decision is made.

To manage this complexity, you should be Unit Testing if not testing-first.

A simple Javascript unit testing framework used for Rails.

js_runit

Visual Studio has a unit test engine, where you can write you unit tests (TDD, Unit Testing, …).
If you need a short (but full!) tutorial to start writing your own unit tests, read A Unit Testing Walkthrough with Visual Studio Team Test by Mark Michaelis.
It also will cover refactoring and code coverage items.

With the latest release of Visual Studio Test System (VSTS) comes a full suite of functionality for Visual Studio Team Test (TT). Team Test is a Visual Studio integrated unit-testing framework that enables:
-Code generation of test method stubs.
-Running tests within the IDE.
-Incorporation of test data loaded from a database.
-Code coverage analysis once the tests have run.
In addition, Team Test includes a suite of testing capabilities not only for the developer, but the test engineer as well.[...]

more info about Visual Studio Products, here Updated: Visual Studio 2008 Product Comparison Guide by Rob Caron

Technorati tags: Visual Studio, Agile, Unit Testing, Refactoring, Code Coverage

I learned an interesting, although in retrospect somewhat obvious, technique from Michael Feathers’s great book, Working Effectively With Legacy Code.

Suppose you have a class that can’t be unit-tested in any automated way because it has side effects or requires user input. Or, in the case of my application, it drives hardware that I don’t actually have readily at hand. NUnit and the like provide input to methods and test the resulting output. How can you run tests on something where the “output” is the motion of a sensor arm, or the “input” is a stream of weather data?

The short answer is that you really can’t, but that shouldn’t stop you from setting up unit tests.

The long answer is that you can hide the troubling class behind an interface, then implement the interface again with a class that can be tested.

To wit, let’s say this is your class now. It moves a data collector probe in 3D space, pulls input of some kind from the probe, and sends output now and then.


public class RealFoo
{
public RealFoo()
{
...
}
public void MoveDataCollectorProbe(int x, int y, int z)
{
...
}
public string GetInput()
{
...
}
public void SendOutput(string x)
{
}
}


This is going to be hard to test, because NUnit can’t see the probe moving around. But what you can test is some file I/O, so you can make a similar class thus:


using System.Diagnostics;
public class FakeFoo
{
public FakeFoo()
{
// probably doesn't do anything
}
public void MoveDataCollectorProbe(int x, int y, int z)
{ Trace.WriteLine ("TRACE: Moving data probe to coordinates x = " +x + ", y = " + y + ", z = " + z);
}
public string GetInput()
{
Trace.WriteLine ("TRACE: Getting input from probe.");
return "Typical Input Here.";
}
public void SendOutput(string x)
{
Trace.WriteLine ("TRACE: Sending output \"" + x + "\" to probe.");
}
}


Obviously the two classes have the same interface:


interface Foo
{
Foo();
void MoveDataCollectorProbe (int x, int y, int z);
string GetInput();
void SendOutput(string x);
}


Then you’d declare that both RealFoo and FakeFoo implement the Foo interface.

At this point, your application code can simply pass a Foo reference around all it likes. While under test, the object factory, or perhaps your own hardcoding, instantiates a FakeFoo. In actual use, or when you have hardware available, you instantiate a RealFoo:


public class FooFactory
{
public static Foo MakeFoo()
{
#if DEBUG
return new FakeFoo();
#else
return new RealFoo();
}
}

You can make the fake object as realistic as you like: randomize output, do intelligent things with the input, perform error checking, or throw exceptions to stress-test the calling code. The main thing is that tools like NUnit will have a fighting chance at picking up the trace stream and interpreting the results as pass or fail. As a bonus, you can do your development when the specialized hardware is unavailable or you’re not ready to run it.

I took another look at RSpec today. I tried to re-write the script using a test driven development method but am finding it harder to use than expected. I think I just have to shift my brain into programming TDD style. I really didn’t come up with any good code, I must have started over at least 10 times. I started with things like it “should accept a parameter –operation that takes either ‘add’ or ’sub’ as it’s value” Then I would read RSpec docs and try to test tha…needless to say it’s a good way to learn about RSpec albeit rather slow. Hopefully I can move my brain to this new style of thinking. It seemed so easy at first but once you try it, it’s actually kind of difficult to do.

Here is the list of requirements I’m going to be working off of.

• it “should accept two parameters, an operation that is either add or sub and a time value formatted at least with one digit. It should also accept two arguments, an in_file and na out_file”
• it “should add or subtract time based on the set operation”
• it “should modify the time by the amount passed in as a parameter of –time”
• it “should read times and all other lines from the first file”
• it “should write modified times and unmodified other lines to the second file”

IF you can get these tests to pass, then you’ve got yourself a working program and RSpec test to go with it as well as some documentation!

Until next time…

Sometimes it’s just nice to have everything you need in your apartment and you can just sit around and code :]

I decided today to try my hand at some unit testing using RSpec, as was suggested at the end of the challenge. I haven’t done much testing before so this seems like a good way to get started. The code I submitted includes a class definition as well as some imperative programming all in one. It would have been nice to separate that out, but this was supposed to be one script that works using the standard library. However, since this is beyond the challenge I’m going to test the class I’ve made, SubtitleTime. You can see it in the pastebin. It takes two parameters, a string in the format of HH:MM:SS,mmm and a ‘modifier’ that is either add or sub.

So starting off with RSpec first you have to ‘describe’ the class. Then you use a function named ‘it’ and write what the class should do. For example:

describe SubtitleTime do
    it "should be able to add another timestamp to itself" do
    end
end

Once that part of the code is written, then you just have to add some filler lines to actually test the code. This code above could be expanded to look like this:

describe SubtitleTime do
    it "should be able to add another timestamp to itself" do
        subtitletime.should add_time(another_timestamp)
    end
end # note: this has been crossed out as after reading more about RSpec, this just doesn't make sense.

Well after having written a few test cases and finally understanding what RSpec is all about, I feel like I should do this the right way and see what happens to my class if I built it using TDD (Test Driven Design). It initially seems very difficult to build up a test suite on an already existing piece of code. So let’s get started with some test cases.

First off we want the class to have a time that, before altered, is equal to the initial time. The code for that would look like this:

describe SubtitleTime do
    it "should have time equal to the initial time before the class is altered" do
        subtime = Subtitletime.new(:initial)
        subtitle.time.should == :initial
    end
done

Then When you run this test with the command <spec subtitletime_spec.rb> it will fail saying there is no class SubtitleTime. Simple. The idea behind TDD is that you make a test case, have it break, then fix it. An occasional (or after each iteration) step of refactoring your code should also be included in this process. So now I’ll make a class SubtitleTime

class SubtitleTime
end

This fixes the first error but we still get another error as the test is run. But that’s all I have time for right now, I was using an ibm guide to rspec. That should make things fairly clear. There is a lot to go over and a lot to do, but I’ll get back to this hopefully sooner rather than later and then finish up this challenge’s extras!

Monday:  Recording of WWI War Guilt podcasts for those signed up.  Review for objective test (40 terms from unit cover sheet, matching, worth 20 points).  Begin reviewing for unit test.

*Outline / plan your answer.

*Answer the question being asked. Read the questions carefully.

*Use specific supporting evidence to support your stated position.  Be sure to answer why the evidence supports your position/helps answer the question.

*Proper sequencing of events important.

*Historiography used only when appropriate.  Not as filler when you don’t know the material.

Tuesday:  Objective test.  We will correct in class once students have finished.  Continue reviewing for unit test.

Wednesday:  Hand out materials for next unit (Mexican Revolution) – written assignment, Underdogs, unit readings.  Notes on defining a revolution.  No unit test for Unit 1 today because it is an Advisory day which means classes are shortened.

Thursday:  Unit 1 (World War One) unit test. Students have been given 4 possible essay questions to study.  On this day I will randomly select 2 using a deck of cards and students must answer one question.

Friday:  Begin notes on Mexican Revolution covering the rise of Porfirio Diaz.

Do you remember the epic fail that bricked the Zune a whole day on the last day of last year, a bisextile year? I described here and here how this error could have been entirely avoided using basic unit testing.

You probably remember (if you read the original post) that I first claimed that it’d take a few seconds to check all possible dates but in fact it ended up taking something like 90 minutes. This week, I come back on unit testing of a very large domain under a time constraint.

So in this previous post, I have been testing the values one by one until I—well, the program—exhausted all input values. This ensures that all values passed into a given piece of code returns the correct result. But while this exhaustive testing increases the confidence one can have in his own code, it has at least two major shortcomings, at least in the specific form I presented.

First, it is exhaustive. At first, exhaustive sounds good because there’s no need to identify particular edge cases because you’re going to test them eventually anyway. But that also means that you’ll test plenty of essentially uninteresting cases as well. Nothing special happened on the Zune until you hit the edge case, that is, the last day of a leap year. So, testing mindlessly all possible cases, while a simple test in this case, is very slow—not that you’re the one pushing bits around either; but 90 minutes is quite a lot of computing power to compute a single test on a single function.

Second, it is not readily capable of detecting infinite loops and exit gracefully, that is, by detecting that it is stuck and report an error. One way of constraining run-time (if you’re using a *nix box of some sort and Bash) is to use the ulimit shell built-in to constrain run-time. For example, typing:

$ (ulimit -t 2  -St 1; date-time-unit-test )

You will get:

CPU time limit exceeded

because when the soft time limit (set by -St 1) is reached, the OS starts issuing SIGXCPU signals to the process that traps it and quits with the CPU time limit exceeded message. If the process is really locked-up (or explicitly eats the SIGXCPU messages) it will terminate with a killed message when the hard time limit is reached.

Using ulimit devolves the responsibility of setting time limits to the shell script that launches the unit tests. This also allows the script to report whether the unit test succeeded or failed, and in case of failure, report the correct error message (SIGSEGV, SIGXCPU, etc.) so that it is easier to isolate the bug the next morning.

Another possible technique is to use watchdog timers if they are available in your test environment. Watchdog timers are timers that count down a certain amount of time and when they reach zero, a specific event is triggered. It may be that the program is crashed with a verbose error message or that a specific signal is sent to the program. To prevent the program from being killed by a watchdog timer reaching zero, the program resets the timer each time it can.

In out Zune example, that would be that the unit test program resets the watchdog timer each time a pair of encode/decode date function are called. If either function gets stuck in an infinite loop, the timer will reach zero and the program is terminated—maybe gracefully, giving it time to report a specific error code.

The POSIX signal SIGVTALRM is meant just to do that. On Linux, you can use signal (defined in <signal.h>) to setup a call back on SIGVTALRM. You would do something like:

sighandler_t my_handler = signal(SIGVTALRM,my_watchdog_handler);

to set the signal handler and timer_create to create the timer, and finally timer_settime to set its characteristics. The body of the unit test launcher would look something like:

#include <stdlib.h>
#include <stdio.h>
#include <signal.h>
#include <time.h>

//////////////////////////////
//
// Handles a signal (in our case,
// SIGVTALRM)
//
void my_sighandler(int this_signal)
 {
  printf("oh noes! got a SIGVTARLM\n");

  // Bash-esque return value
  exit(128+this_signal);
 }

//////////////////////////////
//
// Main program that setups
// timers, signal handler and
// would launch unit tests as
// well.
//
int main()
 {
  // set event-handler
  //
  if (signal( SIGVTALRM,
              my_sighandler)!=SIG_ERR)
   {
    timer_t timer_id;
    struct sigevent signal_event;

    // creates a timer for
    // say, 2 seconds.
    //
    signal_event.sigev_notify = SIGEV_SIGNAL;
    signal_event.sigev_signo = SIGVTALRM;
    signal_event.sigev_value.sival_ptr = &timer_id;

    if( timer_create( CLOCK_REALTIME,
                      &signal_event,
                      &timer_id
                      )==0)
     {
      struct itimerspec timer_specs, timer_out_specs;
      // we created a timer!
      // let us set granularity
      // (and make it repeat only
      // once)

      // initial delay
      //
      timer_specs.it_value.tv_sec=2;
      timer_specs.it_value.tv_nsec=0;

      // repeat delay
      // (zero means no repeat)
      //
      timer_specs.it_interval.tv_sec=0;
      timer_specs.it_interval.tv_nsec=0;

      if ( timer_settime( timer_id,0,
                          &timer_specs,
                          &timer_out_specs)==0)
       {

        // ok, timer is set,
        // let's eat the CPU !
        // (let's pretend the
        // unit test is launched)
        // from here
        //
        while(1);

        // would exit normally
        // if reached at the end
        // of the unit test
        return 0;
       }
      else fprintf(stderr,"error setting timer properties!\n");
     }
    else fprintf(stderr,"could not create timer!\n");
   }
  else fprintf(stderr,"could not set handler!\n");

  return 1;
 }

What you put in the signal handler is up to you, but I would suggest printing something informative and returning a standard-looking error code (such as those here).

When you compile the above code snippet (and use -lrt to link the library that contains the timer calls) to timer-test, you get the following output:

$ timer-test ; echo $?
oh noes! got a SIGVTARLM
154

and 154 is indeed 128+SIGVTALRM. Success!

*
* *

Exhaustive testing can be very long. With the Zune example, we’re lucky because the domain isn’t incredibly vast and computers are quite fast enough nowadays to run calls to a simple function (or a couple of) in a matter of mere hours.

Since you’re not especially interested in finding all edge cases by hand, you could enumerate the test value in some order that pushes somewhat the bad cases in the front of the list. For example, one could first enumerate timestamps that correspond to elapsed whole days, then hours (even with redundancies) then minutes, and at last seconds. The Zune leap day problem would have been caught very rapidly during the first phases of the unit test and everybody would’ve been much happier.