General Reading related to Performance Tuning:
The Goal: A Process of Ongoing Improvement
More technical reading on Performance Tuning:
The Practical Performance Analyst
Web Performance Tuning: Speeding up the Web
Something to relax – but still IT related
Weaving the Web: The Original Design and Ultimate Destiny of the World Wide Web
In order to be able to tune a system it is necessary to establish how the system currently performs. This is done by establishing and running a defined scenario. What this means will be explained shortly. The performance will be established according to the measured performance criteria (as defined previously). Accurately simulate the system usage – the key aspect of this is the definition of a baseline scenario. A baseline scenario defines a number of business processes that are used to emulate real user behavior for the business system.
Scenario
A scenario is based on a number of business processes that are executed by virtual (automated) users according to a percentage user distribution.
User Think Time (think time)
The think time defines a value in seconds a virtual user will wait after completing a User Action before stating the next User Action and is independent of the system performance.
Identifying the think time and user distribution
Think time and user distribution describe what we define as a usage pattern for the system. Lets start with explaining the importance of those two factors on the performance of a system. The think time, as defined earlier, represents the time a user would take from having received a screen to submitting the next request. This time depends very much on the amount of information a user has to process before making and taking a decision on its further action as well as any amount of data that needs to be entered.
When we look at the Petstore application, for example, the time it takes from looking at the home page to clicking on a link will be significantly less than from looking a the address and payment details entry screen to submitting this information. The shorter the think time the higher the number of requests send to the system within a business process and therefore the higher the load on the system. There are different ways in obtaining think time information for a business process, i.e.:
• User interview.
• User observation.
• Logging of user behavior on the system.
When user interview and observation are not possible, i.e. public web site, then a logging of user behavior should be implemented and evaluated.
In cases where this data does not exist, estimations should be made on basis of own usage pattern and/or other data, i.e. the number of submitted orders during a day. We could now argue that the individual thinktime is irrelevant and could be calculated as an average thinktime between transactions based on the whole expected or calculated duration of a business process. Well if no other data is available this might be a solution, but keep the following in mind:
• Individual think times will create a different load pattern on the system than equally spaced out think times. The load pattern observed with individual think times can sometimes be referred to as a Heartbeat pattern, where quick successive requests follow a long period of idle time. The system therefore will need to be able to handle a spikier request pattern.
• Equally spaced out think times will avoid some of the spiky request pattern and enable the system to handle more load in terms of concurrent users than it would in reality. This needs to be pointed out when establishing the maximum number of concurrent users for a system.
The next parameter that determines the performance behavior of a system is the user distribution over the business processes (BP). For example, we have two business processes, one is Browse Books (BP 1) and the other one is Find all books that contain the letter ‘A’ in the title (BP 2). Given a number of concurrent users on the system as 100%. If 50% of all users are executing BP 1 at any one time on the system, the work the system will have to do will differ significantly from having 80% of all users executing BP 2.
Again the distribution of users over business processes can be retrieved on the same manner as for think times.
Information sources that help establishing think times and user distribution over business process can be the following:
• Access logs
• Session logs
• Page Requests per second
• Code execution statistics
• Database analysis
After the goals (KPI’s) have been defined, we need to find a way on how to reach them. The most important point about Performance Optimization or Tuning is to have a valid view on how much of the resources are needed to perform a certain piece of work.
Benchmarks defined by organizations like SPEC and TPC work on predefined types of workloads. Also, the type of system measurements taken is very specific and comprises a certain subset of all available system measurements. Those benchmarks are most useful for comparing different products and architectures.
For our type of Performance Optimization we create workloads that are relevant to the business of the customer. For this, a number of Business Processes will be selected, that are representative of the system usage (see Defining the Business Processes). With this workload we will first measure end user response times and system resource usage. It is important to combine these two types of measurements in order to be able to distinguish between several types of bottlenecks.
Workload breakdown is important for problem analysis, and for future capacity planning under changed or new workloads.
A business process represents a series of user interactions with the system (one user interaction can consist of multiple steps). To be more precise a user interaction is made of a request by the user, i.e. clicking on a link, button or typing in a URL and pressing return. The response will be the screen that is returned by the system. The sequence of user interactions usually fulfills a purpose, i.e. browsing special offers, ordering a book, transferring some money.
Business Process
A business process contains a number of transactions that are performed with a defined user think time between transactions. Usually a business process defines a series of user interactions with a system to achieve a business goal, i.e. entering an order.
User Action
A User Action contains a group of steps that describe a single user interaction with a system. Transactions are defined by a start and end tag, contain criteria for success and failure and are measured in their duration (transaction response time).
Step
A step represents a single client-server request/response. This means that, i.e. a single request and response for a HTML page from a user point of view will usually contain more than one step according to this definition.
Each of the business processes that will be selected need match at least one of the following criteria:
Once the relevant business processes are selected they need to be documented. The format as in the technical questionnaire (see <to be created>) should be used.
It is very important to define each data entry in terms of data range and frequency of change. This should match real system usage.
A very important thing one must do when getting involved in a Performance Optimization (PO) project is to define the scope of the tuning exercise. The scope of the tuning defines the physical (i.e. machines) and logical (i.e. database instance) components of a system that will be investigated and optimized. The three main areas to discover are:
Not necessarily everything that makes a business system is included in a PO project.
Before you can check if any system is performing optimally, you first need to know what you expect it to be doing. Ultimately, this is to satisfy the business requirements for which the application was deployed.
These requirements should be expanded further and form part of the Service Level Agreement (SLA), which defines the nature of the service that the system and application is expected to deliver. Then you need to define a number of Key Performance Indicators (KPIs).
The SLA is an end-user oriented document, using standard business terminology. KPIs are specific definitions of quantities to be measured, that together show that the SLA has been met. The SLA mentions the general targets of the system.
The KPIs define exactly what to measure, how to measure it, and the allowed values. The KPIs are unambiguous so that there are no disagreements over what the target is, what is being measured, and what the system is achieving.
IT Performance Testing and Tuning 