Wednesday, August 20, 2008

Never write software that fails silently!!

One of the tenets of software development is to fail early. The earlier something fails the less costly it is since you don't have as much invested as if it failed later. Visual Studio needs to learn this lesson.

I just spent the better part of an hour trying to figure out why a DeploymentItem attribute on a test didn't work. I had created some new files for the test, added them to the project and created an attribute that should have copied them over. The test didn't fail in the expected manner, because the code hadn't been updated to handle multiple files, but rather because there were no files at all. Setting a break point indicated that in fact the files were not being copied even though the were right there in the project.

After much frustration, I found through cygwin's grep the missing element: the files themselves needed to have their Copy to Output Directory property changed from Never to Every time. Why couldn't anything in the five steps leading up to the test execution have told me there might be a problem?

(And why did Explorer's Search fail to find the files that grep did?!?)

Monday, July 21, 2008

Adventures in firmware development: You learn something new everyday

Ethernet and TCP/IP, the core technologies for the internet, have been around long enough that typically you plug things in and they "just work." So I have been pretty puzzled the last couple weeks trying to figure out why this little Ethernet development kit I'm working with had a 30 second delay on initial start up. Internally, I could see the controller thought everything was up and running, it just didn't see any packets even though the link and data lights would light up.

I contacted technical support for the board and they couldn't reproduce the problem. Hmm. I hate those kinds of issues. After a number of messages back and forth, we found that I was going through the corporate router and they used direct connections to their test PCs. I changed my configuration to talk directly to my PC and the problem went away. Well, this was a step in the right direction.

So, the next step was to talk to our network admin to see if he had any ideas. He immediately remembered a configuration parameter called PortFast on our switches that by default was disabled.

PortFast is an option on Cisco switches to turn STP, short for Spanning Tree Protocol, on and off. I'd never heard of it before but this is what I learned: when you have multiple switches, routers and hubs on a network, it is possible to create a physical loop. Without STP, a physical loop can cause each router to forward packets to the other, ad infinitum, flooding a network with an increasing number of identical packets leading to a non-responsive network. It's basically a DOS attack on yourself. STP is something implemented by switches to detect physical loops so they can filter problematic packets, eliminating the cascades that can take down a network.

STP has five steps it goes through, two of which take approximately 15 seconds each. This is where my delay came from. When you have something plugged into a switch that you know cannot create a physical loop, you can enable PortFast. This bypasses these two steps, changing the start up time from around 30 seconds to almost instant.

The network admin hasn't had a chance to change this setting, but in a number of tests, I'm almost certain this is the problem I was having. I can reliably set things up to keep the router from knowing the board has reset. When I do this, it restarts in a few seconds; the expected behavior. In the end, the problem had nothing to do with the new board and everything to do with the existing router. It happens all the time with my PC too, it's just that the computer takes long enough to boot that the network has finished its protocol negotiation by the time the computer is ready to go.

For more detailed information:

Friday, July 18, 2008

Adventures in firmware development: New task, new tools

Idaho Technology, the company I work for, is experimenting with the idea of using ethernet for communicating with future instruments. Since our hardware engineers have zero experience with networking, and I have some from the PC side, I've recently been tasked with helping them out. I must say, it's been a delightful change of pace from the Windows desktop app development I've been doing of late.

Last time I worked on firmware was 20 years ago. Way back then, to change the the code on the device, I had to use a command line C compiler to generate .hex files on a PC. Then I used an EPROM burner to put it on a memory chip, physically take the memory out of the burner and put it in the device and finally turn it on. Debugging was a matter of examining the source code (and sometimes the generated assembly code) to figure out why it didn't work as expected. Changes to the code involved taking the memory out of the device, erasing the memory using UV light for 30 minutes (I think that's how long it took) and doing the whole, compile/burn cycle over again. The insides of the chip were completely opaque; there was no way to examine what was going on inside the device.

Now, the tools are so much cooler. I have a ethernet development system with an 8051-based micro-controller and programmer sitting on my desk. The programmer is plugged into the PC through a USB port on one side and to the 8051 controller through a ribbon connector on the other. There's an IDE I can use to program, compile and download the new firmware to the chip through. No power cycles on the device, no memory chips to erase and program. And the really, really cool thing: I can set breakpoints, evaluate registers and step through the firmware all from my PC. It's not much different than writing software for the PC itself.

Yeah, I know, these tools have been around for awhile. It's just this is the first time I've had a chance to play with them. Pretty fun stuff.

Tuesday, April 1, 2008

Welcome to the new home of Skylark Software

We're in the process of changing our servers. The old articles will be posted here as I have time to move them over and I have plans, time permitting, for new articles.

Monday, August 28, 2006

How can I tell the difference between a text file and a binary file?

Short answer: You can't. The question does not make sense. A "text" file type is a subset of all binary file types. Let's make the question more generic: how do I tell the difference between file type X and a binary file (where X is any chosen file type (text, MP3, wav, etc.)? Or to put it in a different domain, how do I tell the difference between oak and wood? Now does it make a bit more sense why you can't ask this question? If not, read on...

A binary file contains binary data. Binary data is composed of bytes with values ranging from 0 to 255. Therefore, every file is a binary file. A file of any given type is a binary file which has a specific structure. It is simply a matter of convention and the interpretation of the structure of the data which differentiates one type of file from another. Many file types have specific values which are expected at particular byte offsets within the file. If these values are not found, then the file is not of that type. Of course, just because the file contains those bytes at the expected locations does not ensure the file is of that type, it just gives an indication that it might be.

The text file type typcially does not have this type of structure, an exception being some of the unicode standards. Some of these have an expected byte value in the first two bytes of the file. If these bytes exist, then it's assumed the file is a unicode text file.

All this said, for some uses, it might be possible to define, in a limited way, what it means to be a text file. One definition would be if the file contains anything other than byte values 9 (tab), 10 (new line), 13 (carriage return) or within the range of 32 to 127, then it is not a text file. The downside to this is that it eliminates the use of accented characters and does not include other control characters which might be included in some applications. The definition could be expanded to include the accented characters in the range 129 through 255. However, this now includes most of the range of bytes and might cause some false positives.

The bottom line is every file is a binary file. Every other type of file is a matter of interpretation of the binary data.

Friday, August 25, 2006

Interfaces 101 - Basics

This article discusses the basics of creating an interface, writing a class that implements the interface and some simple uses of the interface. For demonstration purposes, we'll define an interface that simply returns objects from some unknown data structure and has a boolean function to indicate if the end of the structure has been reached. This is commonly called an Iterator. In this article, we'll implement this interface with a class that takes a TList as a parameter in its constructor and iterates over the items in the list. In future articles, we'll probably use this same interface to demonstrate alternate and more advanced ways of implementing it.

Defining an interface

The first step in using interfaces is to define an interface. Typically in the interface section of a unit, it is similar to defining a class. The primary differences are that the interface keyword is used instead of the class keyword, and all that can be defined are functions, procedures and properties. No variable or constant declarations are allowed.

One thing unique about interfaces is they each should have a unique GUID. This identifier is used to find the interface when given an object or interface reference of a different type. While the GUID is optional, it is strongly recommended that all interfaces have one since its absence will make certain functions quietly fail. This can lead to a bit of head scratching. A GUID can be automatically generated in the IDE by pressing Control-Shift-G.

For example:
type
  IIterator = interface
    ['{DFA2FE47-053A-4F5C-BB30-8F2A8C6936EE}']
    function AtEnd: Boolean;
    function NextItem: TObject;
  end;

Creating an implementation

An interface by itself doesn't do anything. There needs to be a class that implements it. This is done most simply by creating a class that descends from TInterfacedObject and is flagged as implementing the interface. There are additional ways of doing this that will be discussed in another article, but for now we'll use this simple way.
type
  TListIterator = class(TInterfacedObject, IIterator)
  private
    fList: TList;
    fPosition: Integer;

  public
    constructor Create(const aList: TList);
    function AtEnd: Boolean;
    function NextItem: TObject;
  end;

constructor TListIterator.Create(const aList: TList);
begin
  inherited Create;
  fList := aList;
  fPosition := 0;
end;

function TListIterator.AtEnd: Boolean;
begin
  result := fPosition >= fList.Count;
end;

function TListIterator.NextItem: TObject;
begin
  result := nil;
  if fPosition < fList.Count then
  begin
   result := fList[aPosition];
    Inc(fPosition);
  end;
end;
The first line of the declaration says we're creating a class of type TListIterator that descends from TInterfacedObject and implements IIterator. It then says we've got two private variables: one will be used to store the current position and the other will store the list object we're iterating over. We have three methods: a constructor that takes the list as a parameter and initializes the two private variables, and implementations of the AtEnd and NextItem functions declared in the interface. All methods declared in an interface need to be implemented somewhere in the class hierarchy at or above the implementing class. Typically, they are implemented in the same class as the one implementing the interface, but they could be implemented in ancestor classes.

Using the implementation

Here are some methods from a test form. First a method to create a list for testing.
procedure TfrmInterfaces101.AfterConstruction;
var
  i: Integer;
begin
  inherited;
  cList := TObjectList.Create;
  cList.OwnsObjects := True;
  for i := 1 to 10 do
    cList.Add(TObject.Create);
end;
Now simply doing some clean-up housekeepping.
procedure TfrmInterfaces101.BeforeDestruction;
begin
  inherited;
  cList.Free;
end;
Now the use of the iterator.
procedure TfrmInterfaces101.btnTestClick(Sender: TObject);
var
  lIterator: IIterator;
begin
  lIterator := TListIterator.Create(cList);
  while not lIterator.AtEnd do
    memoResults.Lines.Add(Format('%p', [Pointer(lIterator.NextItem)]));
end;
Just as if normal class types are used, this method first declares a variable of the IIterator type. Next the variable is assigned a new instance of a class. This is where the first difference can be noticed; the object is instantiated with the TListIterator class but it's assigned to a variable of a different type. Objects that implement an interface, either directly or in an ancestor, can be directly assigned to variables of that interface type. Next there's a loop that uses the two methods of the interface to get each item in the list.

Finally, notice there's no freeing of the iterator. In general, when objects are assigned to an interface there's no need to free the implementing object. The object is freed when the last interface reference goes out of scope. So, in this case, the TListIterator is freed in the compiler generated code related to the "end" statement.

Thursday, August 3, 2006

What are interfaces?

Interfaces are a means of describing a unit of functionality without regard to how it is implemented. They provide a means of decoupling what an object does from how it does it. In some ways, they are similar to a pure abstract class definition.

A great example of a good use of interfaces is the Iterator interface in .NET. It is small and does one thing well: it provides the concept of a list of items without giving any detail about how that list is stored. Various types of container classes implement the Iterator interface, in different ways which are specific to how each container stores its data. Users of the Iterator interface can iterate through a collection of items without knowing if the collection is a linked list, a binary tree or coming from a remote machine through a TCP/IP socket (assuming there's some object which communicates through a socket and implements the Iterator interface).

In Delphi, they can also do two helpful things.

First, since an object can implement multiple interfaces, they can provide some of the useful features provided by multiple inheritance. For example, an object can implement an IStreamable interface, indicating that it knows how to stream itself to some streaming mechanism. It can also implement an IPerson interface indicating that it has Name, Home address and Birthdate properties. Interfaces implementations can also be delegated to properties. This allows a multiple classes with different object hierarchies to implement an interface but keep the implementation in a single helper class.

Second, they can provide a means of automatic lifetime management of objects. There's always the ongoing question of who's responsible for freeing created objects. Some environments, such as Java and .NET, use garbage collection which use various means of determining when an object is no longer used and getting rid of it. Historically, Delphi has said it's the programmers responsibility to call .Free when they are done with the object. Based on this, one best practice says the object which creates another object is responsible for freeing. Another is the owner pattern, the most familiar example is TComponent, where an owner is assigned who's responsible for freeing the object.

Reference counted interfaces are another means. Any object which descends from TInterfacedObject, or which implements IUnknown and incorporates reference counting, can use this method. Basically, it leaves the compiler and run-time system responsible for keeping track of how many references there are to an interface and destroying the implementing object when the last reference goes away.

This can be very handy in many situations. One example is the Factory pattern where the whole purpose is to decouple the creation of an object from the object using the created object. Another example is in threads, where objects may be passed between threads with no clear concept of owner. And finally, in lifetime management of normal objects, it can eliminate just busy-work housecleaning.