Performance test, more than test 性能测试, 不只是测试 (1)

Everytime, when people mention about performance test, some guys would stand out and say: aha, performance test, using loadrunner, collecting performance data, that's it. So, there are many docs talking about how to use loadrunner, and even more documents on loadrunner questions for performance testing interview.

 

If you also think about performance test as these guys, I can only say You know nothing about it!!!

 

From my experience, test is just one of the step when you are trying to do performance analysis, and loadrunner is just a simulator used for testing. For a real performance engineer, he should not only know how's a system's performance, but also know WHY! The system is slow. Ok, why it is slow, why it is not scalable, where is the bottleneck, disk, memory, network, synchronization, a O(n2) algorithm....

 

Here is a list of things a typical performance engineer would encountered:

 

1> Understanding the whole system architecture and the hospots of system

via code and doc reading

 

2> Design test cases to uncover potential performance issues. How well your test cases are based on how well you understand your target system or modules.

 

3> conduct performance benchmarks

 

4> monitor and analyze performance issues to pinpot the bottleneck.

 

5> fix or implement dummy fix for performance issue. Sometimes, you need to verify your assumptions are correct, so, you may change the code and to see whether the issue is fixed.

Under production environment, you may required to tuning system settings for better performance.

 

6> fire bugs and verify bugs.

 

7> document test result with your findings and analysis

 

8> Research and use tools for performance test, such as: loadrunner. Ok, you see loadrunner here. But actually we don't use it. Haha.

 

9> Implement tools for performance test. In many cases, you need to create your own framework for performance test. Even you use some open source simulators like loadrunner, you have to understand its internal implementation, since you may not be able to understand your test result if you do not know what's going on behind the scene.

 

10> Research and use tools for performance monitoring and analysis. Such as: jProbe, perfmon, Memory Analyzer, jConsole...

 

11> Implement tools for performance analysis. Sometimes, you may need to create some simple and easy to use profiling tools by ur own.

 

 

Lastly, I believe a really good performance engineer MUST have passion on knowing the details of any kinds of techniques.