Thursday, October 25, 2007

Unit Testing (1): Overview

First article of a series: Use unit testing with open source Java components, in the environment of Swing UIs, J2EE server components, database access (esp. hibernate), web applications, etc. Unit testing is an important part of Test Driven Development (TDD). Admittedly, I have mostliy ignored this topic up to now. Obviously, the overview I have over my own code was enough to detect, place and correct bugs; working with others' code, detecing errors was usually easy, leaving them to fix it ;-) Also, I was always of the opinion that "my" projects were not applicable for Unit Testing ... complicated UIs, enterprise architectures, complex databases, J2EE deployment, etc. Luckily, others have worked on developing frameworks for most of these purposes.

Test-Driven Development

First write the tests, then the structure of the class to implemented, then run the tests. They'll fail, as there is no implementation yet. So, do the coding, test it regularly, until the tests don't fail any longer ... finshed, sort of.

JUnit

JUnit is the most-used Java testing framework. There are so many good tutorials out there, so I don't see the need to write another one. Read one or more of
  • http://www.junit.org/
  • http://junit.sourceforge.net/
  • http://www.ibm.com/developerworks/library/j-ant/
  • http://www.torsten-horn.de/techdocs/java-junit.htm
In the following, I will use JUnit 4.4. Version 4 changed a lot of the usage of Test Cases and such, using Java 5 annotations.
  • http://www.frankwestphal.de/JUnit4.0.html
  • http://www.mm.informatik.tu-darmstadt.de/courses/helpdesk/junit4.html
  • http://radio.javaranch.com/lasse/2006/07/27/1154024535662.html
  • http://www.instrumentalservices.com/content/view/45/52/
I will also make use of the assertThat statement which was introduced with JUnit 4.4, because it makes much of the test code more readable, especially the error messages.
  • http://junit.sourceforge.net/doc/ReleaseNotes4.4.html

Sample

We write a simple utility for concatenating the textual representation of list elements, with a user-defined concatenation string. Having the list ("Hello", "world", "!"), CollectionUtils.collectionToString(list, ", ") should produce "Hello, World, !".

Unit Test for collectionToString

We start with the unit test.
public class  CollectionUtilsTest {
@Test public void testCollectionToString() {
 final List <String> list1 = Arrays.asList("Hello", "World", "!");
 final String comma = ", ";
 final String result1 = CollectionUtils.collectionToString(list1, comma);
 // we can check the complete string
 assertEquals("Hello, World, !", result1);
 // or the items
 for (String s: list1)
   assertThat(result1, containsString(s));
 // and, all items except that last, need to be followed by ", "
 Iterator  it = list1.iterator();
 while (it.hasNext()) {
   String s = it.next();
   if (it.hasNext())
     assertThat(result1, containsString(s+comma));
 }
}
}
assertEquals() is quite easy to understand, imported via import static org.junit.Assert.assertEquals, and compares its two parameters for equality. Usually, a first parameter should be added that contains the message to print, if the two values are not equal.

Code for collectionToString()

Of course, we also need the CollectionUtils class.
public class CollectionUtils {
// for the moment, we don't do anything
public static String collectionToString(List<?> list, String comma) {
 return null;
}
That's enough to run a unit test ... with eclipse, you just need to start either the test case class (CollectionUtilsTest above) with "Run as JUnit Test", or -- alternatively -- the whole project, which runs all unit tests in the selected package.

First test run

java.lang.AssertionError: expected:<Hello, World, !> but was:<null<
...
  at sbr.jut.demo.CollectionUtilsTest.testCollectionToString(CollectionUtilsTest.java:29)
...

Coding collectionToString

public static String collectionToString(Collection coll, String comma) {
StringBuilder sb = new StringBuilder();
String sComma = "";
for (String s: coll) {
  sb.append(sComma);
  sb.append(s);
  sComma = comma;
}
return sb.toString();
}
And, now, the unit test runs through ... until we start some demonic testing ;-)

Special test cases

Of course, null pointers and stuff need to be taken care of, especially ...
assertEquals("collectionToString(null, null)", "",
CollectionUtils.collectionToString(null, null));
assertEquals("collectionToString(null, null)", "",
CollectionUtils.collectionToString(Collections.EMPTY_LIST, null));
final String result2 = CollectionUtils.collectionToString(list1, "");
assertEquals("HelloWorld!", result2);
final String result3 = CollectionUtils.collectionToString(list1, null);
assertEquals("HelloWorld!", result3);
Nice, we start with the first NullPointerException
java.lang.NullPointerException
  at sbr.jut.demo.CollectionUtils.collectionToString(CollectionUtils.java:18)
  at sbr.jut.demo.CollectionUtilsTest.testCollectionToString(CollectionUtilsTest.java:45)
...
, start fixing our implementation, until we end with ... no NPE!
public static String collectionToString(Collection coll, String comma) {
if (coll==null)
  return "";
if (comma==null)
  comma = "";
StringBuilder sb = new StringBuilder();
...
}
So, that is enough for the beginning ...

Sunday, October 21, 2007

Java programming dynamics

A very good series of articles about Java reflection (how to access a class' members and methods by name), class loading, and bytecode manipulation can be found at the IBM pages (April '03 - June '04):

If you want to learn something about the one the reasons that make Java such a powerful languate, read it. These features are, e.g., used by JEE servers or persistence frameworks to automatically determine the methods and fields to be deployed, for accessing the Java 5 @Attributes (not in the articles, yet comparably easy), and to code or derive proxy classes for the deployed services. Part 1:
  • .class file format
Part 2:
  • Hierarchy of class loaders
  • fields and methods by reflection
  • security of reflection, and how to disable it (e.g., to access private members(1))
  • reflection performance
Part 3:
  • Using reflection for processing command line arguments
Part 4:
  • Inject bytecode into existing methods with javaassist
  • introduce method timing into compiled code
  • Java Aspect Oriented Programming (AOP)
Part 5:
  • Intercepting class loading
  • modifying class bytecode on load
  • introduce method timing into code at load time
Part 6:
  • Code conversion
  • class mutilation made easy
Part 7:
  • The Byte Code Engineering Library (BCEL) (also have a look at ASM, also, as this library tries to correct BCEL's deficits)
  • using "coding constructs" instead of "bytecode assembler"
  • the verifier of BCEL
  • disassembling with BCEL and its graphical display
Part 8:
  • Reflection on performance
  • building a glue class
  • improved performance of code generation vs. reflection
Notes: (1) I know you shouldn't do this, especially not with human men ...

Tuesday, October 16, 2007

Auto-Repeat of Server Calls

Problem: When doing client/server-programming, the client calls a function on the server -- EJB, RMI, Web-Services, etc., in a language such as Java. Sometimes, the connection to the server gets lost, as the server is restarted, redeployed, network was down and up again, etc. Then, the client's call to the server fails; after reconnecting, it should be possible to repeat the call and get the wanted result. Possibly, the reconnection will take some time, if the network has a longer "outing". How can we ensure that the call is done in any case, even if there are transient network/deployment problems? Solution 1: The client's Business Delegate (you have one, haven't you?) calls a "ping()" or another such method before calling the real business method on the server. If the connection is broken, catch the exception, try to reconnect, until such a ping goes through. Disadvantages:

  • doubles the number of c/s calls
  • if the ping succeeds, but the real call fails due to problems in this nano second, you're out of luck
Solution 2: Wrap the call to the server inside a runnable, and repeat the call until it succeeds. Requires a language like Java, that supports runnable concepts. Let's see ... some code, untested, out of thin air.
 protected IServer mServer;

public ResultObject businessMethod(final int intParam, final String stringParam)
    throws BusinessException {
 class BusinessMethodRunnable implements Runnable() {
   private ResultObject result;
   public void run() {
     // do the server call itself, and store the result.
     result = mServer.businessMethod(intParam, stringParam);
   }
   public ResultObject getResultObject() {
     return resultObject;
   }
 }

 BusinessMethodRunnable runnable = new BusinessMethodRunnable();
 // wraps the call, to ensure successful completion
 callMethod(runnable);
 return runnable.getResultObject();
}

protected void callMethod(Runnable runnable)
   throws BusinessException {
 // repeat, until the loop is left with return after a successful call
 while (true) {
   try {
     if (mServer==null)
       // should block until the connection has been established
       connectToServer();
     // do the real call here.
     runnable.run();
     return;
   }
   // a business exception needs to be passed to the calling code
   catch (BusinessException e) {
     throw e;
   }
   // a runtime exception also needs to be passed to the calling code ...
   catch (RuntimeExceptione ) {
     throw e;
     // alternatively:
     // throw new BusinessException("RuntimeException caught", e);
   }
   catch (Exception e) {
     logException(e);
     // connection down, no resources to be freed(?)
     mServer = null;
   }
 }
}

Restoring Word 95 files on Windows 95 after crash

Some of us are still using Windows 95. (Probably none of "us", as I doubt any of those systems is able to surf the net ;-) Mostly on machines that need it ... as nothing newer runs. And, of course, for these systems, there is only Word 95 and companions. Also, these machines still have floppy drives. Not to forget -- most of the users of these systems are a bit digitally ... challenged. Now, there is a user who thought saving files on a floppy disk would save them ... in case of a system/harddisk crash. Not as a backup, but for working on the file. Well. As saving on a floppy takes some time, it's usually not done that often. And computers crash. So, we have an hours old file on a floppy, a rebooted system, and an old version of office that may or may not have auto-recovery files. And if it has those, where are they stored, if not in the working directory? Which is a floppy. 3/4 of an hour telephoning later, the file was found in c:\windows as "Sicherungskopie von ...". Rename it to .doc, and open it ... there it is. Wait ... first, we need to explain, still via telephone, how to make windows show the extension ... otherwise, you can't change its extension. Well, I should just send the bill for nearly an hours work ... probably the same as a new old computer for the user ;-)

Tuesday, October 09, 2007

Avr32 Studio and .d files

Has been some time since I wrote this, so it's high time for publishing it ...!
There seems to be a problem with the Atmel AVR32 studio. It is based on eclipse, to develop C/C++ software for the embedded AVR32 chips.
The following error message is (often) given, when a project is recompiled:
make -k all
src/main.d:1: *** multiple target patterns. Stop.
Build complete for project aixC-BoxFirmware
This usually happens when the previous build was not successful, b/c one ore more files could not be compiled.
The reason seems to be the following line in the automatically generated Debug/src/*.d files:
src/main.d src/main.o: ../src/main.cpp ...
It seems, make is not really able to cope with the multiple main.d main.o: statement ...?
The .d files are responsible for checking all file dependencies, i.e., which files need to be recompiled if one header has been changed.
After a clean, which deletes *.d, a full build works.
avr32-linux-gcc is called with the parameters
-MMD -MP -MF"src/main.d" -MT"src/main.d"
which means:
Generated src/main.d (-MF) with the target src/main.d (-MT), for all user include files, and compile the source (-MMD), and generate phony targets for depended files (-MP).
So, theoretically, only
src/main.d: ...
should appear in the src/main.d file ...
Workarounds:
1. Delete all .d files before recompiling
not nice ... looses all dependency information
2. Fix the generated .d files
The following, a bit obscure sed command can be run inside Project/Debug to fix the .d files.
find . -name '*.d'|xargs sed -ie 's#\.d src/[^/]\+/[^ /\.]\+\.o:#.d:#g'
// do I need .d or .o?
The real problem seems to be s.th. else ... the makefile itself (src/subdir.mk) has a rule for building src/main.o from src/main.cpp, and the .d file contains another rule ... strange version of make ...?
The real problem is the inclusion of C:/home/Projekte/Avr32/aix-AVR32-libs/libserial-0.5.2/src/
which gives the other ":" that raises the error ...
1. use a relative path
2. use make 3.80 ...
This error is usually encountered in other situations:
1. a file name contains spaces (windows makes, mostly)
2. a file name contains : (windows again ;-)
http://sunsite.ualberta.ca/Documentation/Gnu/make-3.79/html_chapter/make_16.html
http://sunsite.ualberta.ca/Documentation/Gnu/make-3.79/html_chapter/make_4.html#SEC40
Have fun,
Sebastian
See also
http://www.avrfreaks.net/index.php?name=PNphpBB2&file=viewtopic&p=315323
You've run into one of our known issues:
Bug #5452
Referencing external folders in managed make projects when using GNU make 3.81 causes build failures. This is caused by GNU make no longer supporting Windows-style path names. GNU make 3.80 works as expected.
The known issues are described in the release notes, which are available on atmel.com. They are also included in the AVR32 Studio Help.
Note that the release notes were not available on the website until quite recently, and the links in the Welcome page appear to be buggy, so I'm not blaming you for not reading them. This will be addressed for the 1.0 release.
Run the cygwin setup.exe and try to downgrade Make to 3.80, that should help.