The blog of Karl Majer…

What is performance?

As a working systems administrators, or programmer, there will be times in which you are called upon to determine the performance of a system, an application, or even an algorithm. While performance testing is frequently time consuming, finding the answer to the question “how fast is it” is neither arcane nor rocket science. All that is required is patience, sound and repeatable methodology, and use of the principles of the scientific method.

Information detailing the scientific method is available on multiple websites. For simplicity Wikipedia’s entries will be used. Wikipedia tells us that the Classical Model of the scientific method is:

• Characterization – Observations, definitions, measurements,
• Hypothesis – explanations of characterization, often theoretical/hypothetical
• Prediction – reasoning from the hypothesis
• Experiment – tests all of the above

The Hypothetico-Deductive model, perhaps more in line with the working professional, is:

• Use your experience.
• Conjecture an explanation.
• Deduce a prediction from that explanation.
• Test

And an even more modern interpretation of the classical model is:

• Define the question
• Gather information and resources
• Form hypothesis
• Perform experiment and collect data
• Analyze data
• Interpret data and draw conclusions that form starting point for next hypothesis
• Publish results
• Retest, often done by third parties

The modern definition would extend the classical model with post experiment work to analyzie, publish, and retest the work done.

Using this information as background, I will be writing a series of articles on understanding performance. These articles will include a few in depth examples to help performance testers understand not only the methods and techniques used, but also the depth and thoroughness required. In the examples, both the hypothetico-deductive model and the modern interpretation of the scientific method will be used.

If you have any examples or questions you’d like covered, feel free to drop me email at kmajer at thisdomainname.com.
>>> Karl

I was trying to move 13G of music between my Powerbook and Windows machines on my gig network at home. It would seem that SOHO switches aren’t  really capable of operating at the advertised gigabit rates, and additionally, Windows XPPro-SP2 is nowhere near optimally tuned straight out of the box. At best, I was only getting close to 120Mbits/sec.

After performing some tuning based on the walkthrough found here: http://www.psc.edu/networking/projects/tcptune/OStune/winxp/winxp_stepbystep.html, I was able to get the Win -> Mac communications up to 260 Mbit/sec. Just for comparison, unoptimized Mac<->Linux rates were 260Mbit/sec using OS X 10.4 and FC 6. I suspect that the limitations of my switch will cap my throughput at that rate.

Naturally, after having done all of the edits instructed in the tutorial, I discovered that dslreports.com offers a utility called DrTCP that drastically simplifies making windows TCP tuning changes.  At a minimum, you will need to restart the network interface that you’ve tuned, if you don’t simply reboot the machine altogether.

For aid in testing during the tuning I used the following utilities:

• Windows – http://www.pcausa.com/Utilities/pcattcp.htm
  •  A copy of the windows binaries are at http://www.genvmunix.com/ttcpzip.exe
• Mac & Linux – http://www.genvmunix.com/ttcp.c
  • The source compiled without any changes with a few warnings doing just a cc -o ttcp ttcp.c on both unix variants.
  • If I can find a netperf client as well I will rerun my testing using that utility.

When time permits, my final round of testing will involve directly connecting the machines together and see what kind of rates I can get by eliminating the switch altogether. For now I have settled with the 260 MBits/sec.

>>>Karl