Leftovers, Leaks and THE LEAK

Today I was looking over the patches I have submitted so far and found that I broke the number 20 today, so I guess it's a good opportunity for a roundup and for putting things into perspective.

Most of the patches had to do with memory leaks and when it comes to coding a memory leak is a pretty bad word, especially in conjunction with the Second Life browser, because the program (or more precisely it's users) seem to have suffered quite badly from the program eating more and more memory with every hour.

I just did a count. 15 of my patches were memory related in one way or another. I did not count exactly, but my guess is that 10 of them were not leaks in the sense that they did not make the program grow over time. They were just sloppy cleanup of one time objects, meaning they did not give up their memory at the end of the program, which is no big issue from a user's perspective, because the memory was required during the runtime of the program and because the operating system will claim the memory a millisecond after the point when a proper cleanup would have given it up. This stuff happens with most programs and those bugs are more leftovers than leaks in that they do not make the memory footprint of the program grow over time.

Then there were a handful of leaks which were neglectible because they just lost a few kilobytes every time you opened or closed a dialog. For example this was the case with the groups dialog, opening that a hundred times would have lost about half a megabyte. Technically this is a real leak ... it does make the program grow every time you do that, but from a user's practical standpoint these minor leaks are irrelevant as well, meaning nobody will be forced to buy more memory for their computers just for being able running the application longer than an hour or two (unless you are opening and closing that dialog every ten seconds).

Which brings me to THE LEAK. The growth of memory over time had a single prime source, which was a leak in a 3rd party library (an open source project called libcurl) which the Lindens were using to fetch data from their servers. As far as I can tell, they were (and are) using the library version from December 2006 and the libcurl folks found and fixed their leak in mid March.

Interestingly the big leak was the hardest to find. It was the second bug I was looking for when I started with the browser source (the first one was the AFK bug) and it took me three full days to nail it.

The reason was partly that it was clouded by leftovers, mini leaks and normal growth of the memory footprint. I first tried to reproduce it more consistently which failed because the program grows to a certain extent over the first half hour as part of the regular design (the memory cache for textures filling up). Took me a day or so to figure it out and eventually resulted in filing an issue to fix regular memory consumption (VWR-733).

Then, when I eventually understood what part of the footprint was "normal" (by design), I still did not know if the stuff I was looking at the leaks log was mainly due to the fact that the program left over parts of it's normal memory consumption or if it was genuinely leaky stuff.

I know now, that most of that stuff in the log *was* actually leaky. The leaks and leftovers I'm finding today are increasingly easy to locate because there is less and less left in the leaks log. It just sticks out. But back then, what I was looking at were logs the size of 50-100MB with nothing but dumped memory blocks.

The reason why I'm writing this is to put things a bit into perspective. It's not that the SL browser is leaking all over. In fact the other real leaks were really neglictible, what made things hard were the leftovers because they cluttered the logs and those logs don't tell which is a leftover and what is a leak.

Which is, why I am currently so religious about fixing all that seemingly irrelevant stuff. Those mini leaks and leftovers may be irrelevant to the end user and with all those shiny new features in the pipeline it's easy to write them off as a secondary or unimportant, and quite frankly, it takes a bit of a deviant coder personality to find pleasure in tracking that kind of stuff down. But sooner or later they are going to come back and bite you int the butt. And interstingly enough, end users seem to have an acute sense about stuff like that, even without knowing exactly why ... just remember the Open Letter project a month ago.

I've been asked a couple of times why I'm investing the efforts here, and quite honestly I'm not entirely sure what's driving me.

Maybe it's because I' thrilled by this project (Second Life), maybe because I love the intellectual challenge. But I guess a big part of it is that I've talked to one person too many who told me they kept crashing every two hours right in the middle of something and that they didn't have or did not want to spend the money for more memory or a faster computer.

I've been following the forum a bit over the last week, and when it came to memory the standard answer was, that you'll need at least 1.5GB or better 2GB on the machine to run SL decently. Most people I know and used to hang out with on the grid however just have 512MB and only a few upgraded to 1GB for the program. I myself bought a new computer because the damn thing did almost swap the hard disk to death on my 512MB laptop and because I thought the low FPS rates were due to the video board.

Go figure ...