Disk Defragmentation -- Useful optimization turned hoax?

I LOOOOOVE defragmentation. I love the thought of my files, my bytes, my bits, on my harddrive is laying in a nice and orderly stream; perfectly aligned and ready to be read right off the magnetic surface of my disks.

So, I was sitting there contemplating the concept of deframentation on a modern harddrive. These days, the disks don't actually report the actual geometry of the disk, instead it maps the "virtual geometry", which is exposed to the OS, to the actual geometry of the disk. This way the disk can easily detect bad sectors on the drive and move and map them to another position on the drive, in case a part of the disk goes bad (which is probably more often than we'd like to know).

So, we power up our favorite defragmentation program and has it move all the files around so that they are ordred perfectly in the "virtual geometry". The problem, as I see it, is that when the disk actually has to read the data, it will be read from the actual geometry of the disk, which could, at least in theory, be scattered all over the disk.

Does this mean that defragmentation is more useful when the disk is new (and has less bad sectors)? Does it even make sense to defragment your disk?