Tuesday, October 16, 2007

Timestamps from Heck: Exposing the Problem

One of the doozies of software engineering - and particularly of testing - is timestamps. Between time zones, time formats, locales, different calendars, this is a particularly rife area for test. This matters even if you do not localize, because times are localized for you.

Of course, if all your clients are in a single office in Boston, this post is not for you! Thank your lucky stars and move on.

Our product is used literally worldwide - 1/3 of our users are in Europe, we have a strong South American population, and recently Russia and New Zealand had quite a spike in interest. On top of that, we're a sync product, so timestamps matter.

Let's look at one simple use case: "A file was modified after the user's last sync." So, how do times enter the picture?

1. "User's last sync" is a time, measured to the second. This comes from the server.
2. "file was modified" means the file has a modified time set on the client machine to whatever specificity the client allows (we're usually on Windows XP SP2, so we'll say to the second).

All is well and good except now we have to compare these two times and we have to be very very sure we're comparing apples to apples. So what are our scenarios:

1. client and server are in different time zones
2. client and server use different time servers that don't quite match (say, 12 minutes off)
3. client and server use the same time server but are off within the allowed margin of error (usually this is between 2 and 10 minutes)
4. client and server use different time formats (say, MM/DD/YYYY versus DD/MM/YYYY)
5. client and server use different calendars (say, Gregorian versus Julian)
6. client and server use different time formats (say, 12 versus 24 hour clock)
7. client and server cross the date line (this is a fancy version of different time zones)
8. client and server measure wtih different specificity (say, one to milliseconds and the other to seconds)

So now we have a basic set of time issues that we have to look for in any time-sensitive operation in our data. That's a lot of testing, so we're going to talk about how we've combined these into as few distinct test cases as possible.

P.S. I'm not going to tell you how we solved this (that's part of the secret sauce, but suffice to say that details have been changed to protect the innocent). The point is more that we have to test it.

No comments:

Post a Comment