I kept notes on what I was doing during the sprint in Iceland. I wanted to clean them up and publish it. This serves as a review for me of what my involvement was, and also is meant to allow other people to see what sorts of things go on at a sprint.

Monday

• Went through the Python tracker and collected a list of performance-related patches in it. This was the first task I did, so that other folks could start working on getting these already existing patches applied.
• While doing the above, I realized that it would be nice to have a "Performance" group in the tracker, so that over the long term we could more easily track and apply performance-related patches. This should make it easier for the community as a whole, over the long term, to keep performance-related updates applied.
• Discussion at the sprint mentioned that there were some easy optimizations that could be done to bitwise operators. I captured this and added a tracker item to keep track of this and ask for feedback on it.
• Urging Steve Holden to discuss the 2.4 to 2.5 pybench slow-down he was seeing. It turned out that some of this was related simply to how the tests were run, but there was obviously a big slow-down in exception handling which Steve hadn't discussed during the kick-off meeting.
• Running a number of pybench benchmark tests between 2.4, 2.5, and trunk. Throughout the week, pybench and measuring test results was an ongoing issue.
• I looked at Gentoo to see if they had any more extreme compiler flag settings that we might want to look into for better performance. I didn't find much in a quick look though.
• Helped Richard Jones run some performance tests to get an idea how much the Zombie Frame patch helped performance. In the end, he decided to add it in, with a number of changes based on the performance testing.
• Tested out an idea about switching Python "integer" types to use 64 bits instead of 32 bits on 32-bit platforms. In the end, it would have been a major pain to do, and while it would have resulted in 34% performance improvements for apps that use integers between 32 and 64 bits, it would have been around a 10% slow-down for apps that only use 32-bit numbers. Decided not to do it for those reasons.

Tuesday

• I continued testing of the 64-bit integer work I was doing the end of Monday and came to the conclusions mentioned above.
• Spoke with John and Martin about new buffer objects and using them for network input. The python memcached system could benefit from something like this because currently it does byte-by-byte reads of certain data because of the lack of a buffer.
• Spoke with Runar about his work to make versions of integer conversion that could take an offset and end point for the conversion. I had a few ideas, imagining that this would be a camels nose under the tent if introduced, where people would want that ability in many other places. Suggested that something like Martin's "byte buffer" could make the need for this in the int() constructor obsolete.
• Contacted Marc Andre about there being two versions of pybench in the SVN repository, and asking if one could be deleted to reduce confusion. He said no.
• Wrote some messages to the python-dev mailing list about the 64-bit integer work I had done above. We pretty much all agreed that it was a bad idea, but I wanted to have the discussion recorded and open it for feedback.
• Wrote a blog post which was a recap of the first day of the sprints. I wanted to write recaps of every day to try to get more involvement, and possibly as a PR/marketing move to let other people know what was going on and open the possibility of it happening more frequently. If this is a win, having it happen more often would be a huge benefit to the community.
• Spent some time looking at the try/raise/except slowdown in 2.5a2 (from 2.4.3). Just started looking into this on Tuesday, but it was a 60% speed decrease as reported by pybench, and nobody had looked at it yet. I spent much of the rest of the day looking at this.

Wednesday

• Another blog post doing a recap of day 2.
• Finished the discussion about 64-ints in Python, and recorded the results.
• Wrote a message to python-dev about the slow exception handling I was looking at on Tuesday, and got some feedback. Worked with Brett Cannon, who wrote the new exception code in 2.5, and Kristjan on profiling the current code to see what was happening. Spent most of the day looking at this, and started looking at implementing the changes based on the investigation. However, Richard Jones came over and said he was free and since he's more familiar with doing the work involved he took over.
• I worked with Richard in a support role by enhancing the tests so that when he was done with the changes we could ensure that we could detect any problems and resolve them.
• Wrote a report of the sprint for Linux Weekly News.

Thursday

• The first half of the day was a trip to the Blue Lagoon for the sprinters, sponsored by EWT.
• Went through the Python tracker and marked the performance-related patches in the new "Performance" group.
• Continued discussion with Runar about the integer conversion that I started on Wednesday.
• Did some pair-programming with Richard Jones on the new exception code.
• Made sure that the exception tests tested the heirarchy of exceptions. Implemented these tests, then Brett noticed and told me that another test already did that check. Fixed a problem in that test where it was not checking the VMSError exception.

Friday

• Working with Richard and Georg on the new exception code. Now that the code was compiling, I spent much of the day running the test suite and tracking down the errors it was reporting, finding what they really meant, and working with Richard and Georg to get them fixed. Spent most of the day doing this.
• I spent some time in the afternoon trying to see what could be done about highly variable results we were getting in pybench. Pybench has a mode where it will compare one run with another, which is meant to test the performance of one version against another. However, comparing a version against itself resulted in performance differences of up to 30%, even when running on an idle system.

Saturday

• I spent the whole day, starting at 5am, working on getting some better benchmarking numbers. I compared different timing mechanisms (time.time, time.clock, resource.getrusage) to find the one that seemed to work the best. I also compared short versus long run-times and other parameters. I ended up coming up with a mechanism for doing tests that produced less than 1% variance when running on my Linux test system from one run of the test to another. Spent most of the day implementing this after running my test methodology by Tim Peters and getting a thumbs-up. We had to vacate the room by 5pm though, so I didn't get it completed before we had to leave. I plan to finish this up after the sprint.

There you have it. It was a ton of work, and some of the things I tried like the 64-bit ints didn't pan out. I'm reminded of a saying: "If everything you try works, you aren't trying hard enough." I do feel like I made some good progress though. I think the most significant work I did was that relating to exceptions (which was mostly management rather than technical work) and that relating to benchmarking. Having a stable set of benchmarks would be a big benefit to the Python community.

It was a crazy, long, week. Because of the sun I was up every morning between 5am and 7am, putting up blog entries and getting started on things before the sprint started at 9am. Most days ended between 8 and 11pm. A lot of work was done, but a lot more was still on the plate to do. It was extremely productive over all though.

I had a great time though. As Richard Jones said though: "I think I'm going to need a vacation after this vacation." It's now beginning of work Tuesday, and I've spent much of the last day finishing up blogging and reports related to the Sprint. The primary thing I have left on my list is to complete the benchmarking. Probably will do that either tonight at Hacking Society or this weekend at bazcamp if I can't get it done tonight.
(Post Reply)