Monday 26 October 2015

Database Performance testing using JDBC connection through Jmeter

Follow below steps to configure the Jmeter to connect your database.

STEP 1: Open a new Test Plan and add a Thread Group in it.

STEP 2: Add a 'JDBC Connection Configuration' from 'Configuration Element' menu into the Thread group.


STEP 3: Configure the 'JDBC Connection configuration' with respective database/tables as described below.

*Variable Name : We need to create a name for the JDBC pool. So that we can reference the pool name while executing a SQL query.

*Pool Configuration : Need to provide the number of concurrent connection in 'JDBC POOL'. (we used just 5, which means we can run just 5 parallel users only). Need to specify the pool timeout value too.

Database Connection Configuration : This is the place we need to provide our database name, host and port.
The access credentials needs to be added into it.

JDBC Driver Class:
Jmeter does not come with mysql JDBC driver. We need to download the latest JDBC mysql driver from Oracle site and hold a copy in Jmeter/bin folder. Mention the driver name in this column.
Only if we have a valid driver name, we can execute the query.

STEP 4 : JDBC Sampler
Add a JDBC sampler from ADD > Sampler > JDBC Request.




STEP 5: Add the SQL


* Give the sampler a valid name to be referenced in results

* Provide a valid Variable pool name, which we created in STEP 3 ( 'JDBC POOL')

* Type your SQL Query in the space provided.


STEP 6: Add a View Results Tree to observe the results.


STEP 7 : Para-metering the SQL

Create a list of parameters to be used in SQL in a 'User Defined Variables' configuration.


Refer the above created parameters in the SQL as required.


STEP 8: Execute the Jmeter


Configure the Thread group with number of Threads. These value give the number of Vusers run concurrently. 

Please make sure these value does not exceed the pool configuration connection count 5(STEP 3)


Now you can observe the result of SQL in the 'View Results Tree' node.

Friday 4 September 2015

SSL Certificate Conversion - Java Keystore(.jks) into .p12 format

Below Keytool command can be used to convert .jks certificate format into .pem format. Where myapp.jks is your source file & myapp.p12 is your destination file.

keytool -importkeystore -srckeystore myapp.jks -destkeystore myapp.p12 -srcalias myapp-dev -srcstoretype jks -deststoretype pkcs12

If your certificate is password protected you can use below command for conversion:

keytool -importkeystore -srckeystore myapp.jks -destkeystore myapp.p12 -srcalias myapp-dev -srcstoretype jks -deststoretype pkcs12 -srcstorepass pass123 -deststorepass pass123

How to Configure SSL Certificate in LoadRunner?

Configuring SSL Certificate in LoadRunner:

Right click at Action panel & choose “Add files to script” option & select SSL certificate and add it in script. Then use web_set_certificate_ex function to set the certificate.
Example:
web_set_certificate_ex(
"CertFilePath=certificate.pem",
"CertFormat=PEM",
"KeyFilePath=certificate.pem",
"KeyFormat=PEM",
"Password=Password",
LAST );



Creating Web Services Script in LoadRunner using WebService/Web(http/html) Protocol

Some Important Definitions:

Web Services:
It is a web application components which can be published & used on web.
Web services do not require the use of browsers or HTML.
In Web Service XML is used to tag the data, SOAP is used to transfer the data, WSDL is used for describing the services available and UDDI is used for listing what services are available.

XML: EXtensive Markup Language
XML is designed to describe the data & act as a tool for carrying information.

SOAP: Simple Object Access Protocol
It is a protocol for accessing web services.
Soap is a format for sending messages & used for communication between applications.

WSDL: Web Services description Language
It is an XML document which is used to describe & locate web services.
It specifies the location of the service and the operations (or methods).


Data requirements:

  • Sample Request & Response XML.
  • End Point URL(Host) where request xml to be posted. E.ghttp://webservice.nam.nsroot.net/
  • WSDLfile to get the information & soap action for all the existing operations/methods.
  • Operation details which needs to be invoked like “GetLoadDetails”.
  • SSL Certificate(also known as a Web Server Certificate, Secure Server Certificate, and Digital Certificate) & password if the host is SSL secured.
Two different ways of creating Web Services Script:

We can do web services scripting in two ways:
  • By using Web Services Protocol.
  • By using Web(HTTP/HTML) Protocol.
Steps for writing script:

  • Open the Vugen and select Web Services protocol.
  • Click on Import SOAP on top navigation bar and select request xml file.
  • Click LOAD and provide End Point URL(Host) & Soap action then press OK.
  • Soap request will be created with the step name “SOAP Request”.
Error Handling & Validation:
lr_xml_find and lr_xml_get_valuesfunctions can be used to validate the page. If-Else statement can be used to validate the resposne.
Example:
lr_xml_find("XML={response}",
"Query=//SOAP-ENV:Body/*/LoanAmount",
"Value=2000",
LAST );
lr_xml_get_values("Xml={response}",
"Query=//SOAP-ENV:Header/*/UUID",
"ValueParam=UUID",
LAST);
Error Handling:
if (strcmp(lr_eval_string("{Response}"), "0") != 0 )
{lr_end_transaction (“Submit_Request", LR_FAIL);
LR_EXIT_ITERATION_AND_CONTINUE;
lr_error_message(“Invalid Response"));
return 0;
}
else
{lr_end_transaction ("Submit_Request ", LR_PASS);
}


Writing Script in Web(http/html) protocol:
We need to create Custom request using Web protocol & keep the xml request in the web request body.
Example:
web_custom_request("Weather SOAP Request",
"URL=http://www.webservice.nam.nsroot.net/GetLoanDetails",
"Method=POST",
"TargetFrame=",
"Resource=0",
"RecContentType=text/xml",
"Referer=",
"Mode=HTML",
"EncType=text/xml;charset=utf-8",
"Body=<<Your Complete XML Request>>"LAST);

We can capture complete response or any particular value using web_reg_save_paramfunction.
Check point can also be added using web_reg_findfunction.
If-Else statement can be used for error handling check.

Examples:
web_reg_find("Text/IC=ResponseCode>0<",LAST);

web_reg_save_param(“Complete_Response",
"LB=",
"RB=",
"NOTFOUND=warning",
"SEARCH=ALL",
"ORD=1",
LAST );



Virtual Table Server(VTS) Enhancements in LoadRunner 12.00

In LoadRunner 12.00, the VTS enhancements include multi-instance support, TruClient native support, sample data, etc. Another small handy feature is supporting database APIs in different protocols, which can be used as an alternative to VTS.

Multi-instance Support
The old 2.x version of VTS is a Windows application, and users can run multiple instances on a single machine. Each process will listen to a TCP port, and each will serve one data table. The VTS in LoadRunner 11.52 is web-based and run as a Windows service, however it can only serve one table per machine. We added this feature to the new VTS in LR 12.00 in response to customers' requests.

There are two ways to launch multiple instances of the VTS on the same machine - from the command line, and from the web UI.

For the VTS instances that are running on one machine, there are two types, which have some slight differences between them:
  • Main Instance: This is the instance that always run when the VTS service starts. It behaves the same as the VTS in LR 11.52. There can be only one main instance per machine, and user can create additional named instances from main instance UI.
  • Named Instance: There can be multiple named instances, as long as they have different names and ports, both of which should be unique across all instances.
The differences are:
  1. A named instance will only use one port, and the port is for both API access and Admin UI display.
  2. You cannot create other named instances on this Admin UI.
  3. The title of the Admin UI changed to ‘VTS - <Instance Name>’, so that the user can easily identify the instance from the tab, as follows:
p1.png

Here are some short descriptions on the two ways to manage instances.

1. Command-Line Mode
The command tool (vtscmd.js) is a JavaScript file that runs with node.exe, which can be found in the same folder. The following is the screen of the command mode.
p2.png


In the new version we will allow users to run the following commands:

  • node vtscmd /start /port 4001 /name <InstanceName>
  • node vtscmd /import file1.csv /port 4001 /delimiter \t

The first line starts a new instance and the second line imports the data file into the VTS table that listens on the same port. "/delimiter" is optional and by default uses ",".  The VTS Windows service should already be started when you run commands, and the command tool should be used on the same machine that runs the VTS service. All of the commands will communicate with the VTS service, so after launching the instances, they can survive the user logging out and machine restarts.

There are demo .bat files in the VTS installation folder, to show how to launch new instances of VTS and how to stop them. You can find them under folder "\Program Files\HP\VTS\web\samples". One limitation of this release is that when you try to stop the VTS instances, the process may still be running until you stop the VTS service. If instances are still running, it will block you from creating another instance with the same name or port. To make the ‘stop’ action take effect, you can restart the Windows service, with the command “net stop vtsservice” followed by “net start vtsservice”.

2. UI Mode
The user can also manage VTS instances from the main instance’s UI, with the File menu Options -> Other Instances. There are two dialogs, one to start new instances and the other to stop instances.
p3.png

Now one machine may run multiple instances of VTS, including one main instance and multiple named instances. In 12.00, API access of all instances will be enabled or disabled together, and only from the main instance UI. The following UI has been updated to reflect the change. Notice the tooltip indicates that "Enable/Disable API Access" action will affect all running instances:
p4.png

One more tip - the VTS runs on the node.js engine, and is written in JavaScript. Therefore, if you feel curious, you can open Task Manager to check how many node.exe processes running, to find out how many VTSs are running.
  
VTS Support in TruClient
In LR 11.52, the user needs to write VTS API calls in a C function, and then use it in TruClient scripts, which is not very convenient. As of 12.00, VTS JavaScript APIs are available in TruClient which can be used directly in TruClient steps. You can add "Evaluate JavaScript" step in the TruClient Sidebar, and add some code similar to the following:
TC.vtcConnect("localhost",8080,"myVTSAlias");
…
TC.vtcGetCell("columnName1",1, "myVTSAlias");
TC.vtcAddCell("columnName2", 1, "myVTSAlias");

Some Other Improvements
Sample data
Sample data has been added to VTS, for example, Customers and Employees, so that users can create sample scripts with data out-of-the-box. Just click the menu Options -> Import Samples to import the sample data table. If you have your own csv files and want to access them often, you can also put them under the samples folder, and the files will also displays on the menu.

Installation
The installation package contains both 32-bit version and 64-bit versions. If you have 64-bit windows, it will install the 64-bit version automatically, otherwise it will install the 32-bit version. The 64-bit version is expected to be more scalable because it uses 64-bit node.js. If you want to install VTS silently, you can run a command like "SetupVTS.exe /s /a /s". There’re actually 2 operations here – extraction files and then installation. Hence the “/s” appears twice, one for each stage.

Step Toolbox
The VTS API can be found in the VuGen Toolbox, so if you don't want to remember the long API names, you can drag and drop from the toolbox to use them.

Database API Support
Although database APIs have nothing to do with VTS, they can be used to make calls to databases in your script, allowing you to use databases as the shared parameter server. Before LR 12.00, there were several database APIs in the Web Service protocol:
  • lr_db_connect
  • lr_db_disconnect
  • lr_db_executeSQLStatement
  • lr_db_dataset_action
  • lr_checkpoint
  • lr_db_getvalue
But they can only be used when web service protocol is used. As of LR 12.00, we migrated them to all of the other C language protocols, e.g. Web HTTP/HTML, Flex, etc. You will notice that they appear on the Step Toolbox of VuGen, so you can design the database steps just as you did in the Web Service protocol.

Some More Information
There are some trouble-shooting tips, not necessarily related to LR 12.00 but worth mentioning here.
  • Some scripts use "lrvtc_update_message_ifequals" heavily and users have noticed that the API becomes slow when there is a lot of data. The cause is there is no index on the column. Just like databases, when there is no index on the column, checking uniqueness of a value will scan all the data. The solution is to add an index to this column. This can be achieved by either the Admin UI or lrvtc_ensure_index API in the script. Note that deleting the entire column will also delete the index, so the next time you create a column of the same name, you need to build index again.
  • If you wants to trouble shoot issues with VTS, you can check the log files. To troubleshoot remotely, you can also read the log file from the web URL http://<server>:4000/data/diag/log. Accessing the log remotely is by default turned off, but you can turn it on by changing the setting "enableDiag" to true in the configure.json file and restarting the service.
  • One common misunderstanding is the return value of the lrvtc_connect API. Unlike another similar API vtc_connect, which returns a handle, lrvtc_connect return the status code of connection. So vtc_connect returns non-zero when connecting successfully, while lrvtc_connect returns zero upon success. 

Virtual Table Server(VTS) in LoadRunner 11.52

With the release of 11.52, HP has released a whole new VTS, with some enhanced features in LR 12.00. The new VTS APIs is backward compatible and can be used in all protocols that uses C language as well as TruClient. You can see the new console UI in the below screen:

VTS2.png

Installation:
The new VTS consists of two parts, the vuser side is tightly integrated with the LoadRunner, and is installed with 11.52. The VTS server side needs separate download and install.

If you want to use the new VTS, you can take the following steps to install the new VTS and experience it:
  • Install 11.52 on all load testing machines, particularly, the VuGen and LG machines.
  • Install VTS server on a server machine, it can be a separate machine, or any of existing load testing machine, depends on how much load VTS service will have. You can either install 32 bits version or 64 bits version, depends on your OS type. After installation, the service will start automatically and you can browse to the VTS console UI. The default port is 4000, therefore the URL is something like http://localhost:4000. You can specify a different port during the install or afterwards.
  • On the console UI, you can import data by uploading a CSV file, or typing in a connection string to import from database directly. The following screen shows the UI of importing data&colon;VTS3.png
  • Create a C language script, like Web Http/Html script. Add VTS API in vuser_init method to connect to the server, like the following:
int   rc = 0;
char  *VtsServer = "192.168.0.100";
int   nPort = 8888; 

PVCI2 pvci = 0;
vuser_init()
{
    pvci = lrvtc_connect(VtsServer,nPort,VTOPT_KEEP_ALIVE);
    lr_log_message("pvci=%d\n", pvci); 
    return 0;
}
  • In Action.c, add some code to retrieve one column from the server and remove from the server at the same time. You can run the action multiple iterations, and it will retrieve one:
char * cname; //column name
int ret;

RetrieveMessage()
{
    lr_start_transaction("retrieve_column");
    rc =  lrvtc_retrieve_messages1("cname",";");
    lr_log_message("lrvtc_retrieve_messages1 rc=%d\n", rc);
    lr_log_message("cname is %s", lr_eval_string("{cname}"));
    lr_end_transaction("retrieve_column", LR_AUTO);
}

Action()
{
    RetrieveMessage();
    return 0;
}

  • In vuser_end.c, add API call to close the VTS handle:
vuser_end()
{
    lrvtc_disconnect();   
    return 0;
}

  • You can test it in VuGen and run it in Controller. Don’t forget to turn on the API access port before you run the script. Like old VTS, the new one use 8888 as the port for API to access. It is turned off by default, so you need to browse to the administrative UI to enable it.

LoadRunner: Performance testing Oracle via JDBC Connection

This Post will give you brief introduction to design a method for testing Oracle via JDBC connection to simulate application interaction with the database.

Understanding JDBC

The Java Database Connectivity (JDBC) API is the industry standard for database-independent connectivity between the Java programming language and a wide range of databases—SQL databases and other tabular data sources, such as spreadsheets or flat files. The JDBC API provides a call-level API for SQL-based database access.
The JDBC Thin Driver used in this method is a JDBC Type 4 driver that uses Java to connect directly to Oracle via the TNS protocol. This driver is written completely in Java and is hence platform independent and provides the best performance of all JDBC driver types.
JDBC - Type 4 Driver - Native-Protocol Driver

Connection String Examples

The following URL connects user "ray" with password "password" to a database with service orcl (Important: see more on services) through port 1521 of host myhost, using the Thin driver.
jdbc:oracle:thin:ray/password@//performance.net:1521/orcl
This URL connects to the same database using the the OCI driver and the SID inst1 without specifying the username or password.
jdbc:oracle:oci:@performance.net:1521:inst1

LoadRunner Oracle via JDBC (Java VUser)

As JDBC is widely used and documented in the Java world, we can use the Java Vuser to use the Java JDBC examples directly in our Vuser script.

Requirements

In order to script an Oracle JDBC connection you require
  1. JDBC Connection String
  2. JDK
  3. Oracle JDBC Drivers

JDK Path

As a Java VUser, the JDK location must be set. This is required to be the same for all load generators as this is a run-time setting for the script rather than the generator itself. In the example (See Figure 1) the JDK path is set to “C:\JDK\”. Note that this JDK directory must contain a “bin” directory that contains the javac.exe.


Script Type

The script type used for Java is the “Java Record Replay” or whatever you have a license for.
LR Record Replay

Database Drivers

To establish a connection to the database, we are obviously using JDBC. In this example we will be using the “Thin” drivers as they are considered to be smaller and faster than the OCI drivers, and doesn’t require a pre-installed version of the JDBC drivers. This is particularly important for managing the Oracle JDBC drivers across multiple machines. With the Thin driver, we can include the Oracle thing JDBC driver JAR files directly in our script to make them portable and not have to worry about installing them on each and every load generator.
Oracle Drivers Included

Scripting

Java vs. C

The JDBC connection script uses a Java VUser as opposed to the regular C script. As much as it would be simpler to stick with C for familiarity, as JDBC is a Java API, it made sense. There are a few small changes with Java script when compared to a C script
  1. vuser_init and vuser_end are contained in the action script
  2. Every function is different than in C (lr_start_transaction(); becomes lr.start_transaction();)
  3. You now have to worry about JDK and Class path run-time settings
It is important to consider these elements, especially point 3. The JDK run-time setting must specify the JDK location path. This path will have to be the same on ALL load generation machines.

Connecting to the Database

There are two steps in a Java to connect to an Oracle database via a JDBC connection. The first step is to dynamically load the Oracle JDBC driver class.
// Load Oracle JDBC Driver
Class.forName("oracle.jdbc.driver.OracleDriver");

The second step is to make the connection to the database using a JDBC connection string, username and password. The following example is connecting to the performance.com host on port 1521 and the database SID of LOAD.
// Specify the JDBC Connection String
String url = "jdbc:oracle:thin:@performance.com:1521:LOAD";

// Connect to URL using USERNAME and PASSWORD
connection = DriverManager.getConnection(url,"Perf",lr.decrypt("perfload"));

Of course being Java, we have to do all the work and specify what we would like to happen if any of the steps fail. The following is an example of an init function that is designed to connect to the database and gracefully abort the Vuser if the database connection cannot be made.

// Create global connection variable
private Connection connection;

// VUser Init
public int init() throws ClassNotFoundException, SQLException {

  try {
    // Load Oracle JDBC Driver
    Class.forName("oracle.jdbc.driver.OracleDriver");
  } catch (Exception ex) {
    // If driver load is unsuccessful
    lr.log_message("Database Driver not found");
    lr.abort();
  }
  try {
    // Specify the JDBC Connection String (jdbc:oracle:thin:@HOST:PORT:SID)
    String url = "jdbc:oracle:thin:@performance.com:1521:LOAD";
    // Connect to URL using USERNAME and PASSWORD
    connection = DriverManager.getConnection(url,"perf",lr.decrypt("perfload"));
    lr.log_message("JDBC Connection Successful");
  } catch (SQLException e) {
    // If Connection Failed
    lr.log_message("Database Connection Failed, Please check your connection string");
    lr.abort();
  }
  return 0;
} //end of init

Database Query Function

In an effort to make scripts as easy to read and understand as possible, I have written a database query function that will return a pass or fail for the currently open transaction. This function performs a Database Query with the parsed in query. The Java code to execute a query against out JDBC connection is as follows.
String SQL_QUERY = “SELECT * FROM TABLE”;
Statement stmt = null;
ResultSet rset = null;

connection.setAutoCommit(false);
stmt = connection.createStatement();
rset = stmt.executeQuery(SQL_QUERY);
rset.close();

The results of the query can be printed using the following code. Note that logging / printing of results should be avoided in a performance test script as takes additional time and processing power to log something that we should not need to look at. This is provided for debugging purposes only.
while (rset.next()) {
  lr.log_message(rset.getString(1));
}

Being Java of course we have to provide a very verbose bit of code to get our errors handled. The following is the entire database_query function.
public int database_query(String SQL_QUERY) {
  Statement stmt = null;
  ResultSet rset = null;

  try {
    connection.setAutoCommit(false);
    stmt = connection.createStatement();
    rset = stmt.executeQuery(SQL_QUERY);
    // The transaction can be set to a Passed Status
    lr.set_transaction_status(lr.PASS);

    // Print the results (For Debugging Only)
    /*
      while (rset.next()) {
        lr.log_message(rset.getString(1));
      }
      lr.log_message("SQL Query Executed Successfully");
    */
    rset.close();
  catch (SQLException e) {
    // SQL Query has failed
    lr.log_message("Caught Exception: " + e.getMessage());
    // The transaction must be set to a Failed Status
    lr.set_transaction_status(lr.FAIL);
    return 1;
  }
  return 0;
}

Scripting a Scenario

Through the use of functions such as database_query, our actual script can be kept very clean. An example that performs three SQL queries on the database with a 5 second think time is below. Note that the example shows SELECT statements, however any SQL command could be used.

public int action() throws ClassNotFoundException, SQLException {

  // Database Query Example 1
  lr.start_transaction("Database_Query_1");
  database_query("select BANNER from VERSION");
  lr.end_transaction("Database_Query_1", lr.AUTO);

  lr.think_time(5);

  // Database Query Example 2
  lr.start_transaction("Database_Query_2");
  database_query("SELECT * from POST_DETAILS");
  lr.end_transaction("Database_Query_2", lr.AUTO);

  lr.think_time(5);

  // Database Query Example 3
  lr.start_transaction("Database_Query_3");
  database_query("select * from users");
  lr.end_transaction("Database_Query_3", lr.AUTO);

  lr.think_time(5);

  return 0;
} //end of action

Verifying the Response

In database performance testing, it is not commonplace to verify the SQL results other than to ensure that the query has been successfully executed by the server. The Java executeQuery function utilised by thedatabase_query function will return an exception if the query is not successfully executed. The db_query function will return the first 10 records from a result set. To get the full SQL results parsed across the network, you will need to process them with the “getString” function of the resultset. In the case where keyword type checks would want to be performed on the SQL results, the same code to print the response could be used to check the results one by one with some form of validation.

Example Script

Here is the full example script for you to try. Please feel free to submit corrections or improvements.
/*
 * LoadRunner Java script.
 * Description: Oracle Database Testing via JDBC
 */

import lrapi.lr;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.*;

public class Actions
{
    // Create global connection variable
    private Connection connection;

    // VUser Init
    public int init() throws ClassNotFoundException, SQLException {
        // Initialize DB connection
        //connection = null;
        try {
     // Load Oracle JDBC Driver
            Class.forName("oracle.jdbc.driver.OracleDriver");
        } catch (Exception ex) {
     // If driver load is unsuccessful
     lr.log_message("Database Driver not found");
     lr.abort();
 }
 try {
     // Specify the JDBC Connection String (jdbc:oracle:thin:@HOST:PORT:SID)
     String url = "jdbc:oracle:thin:@performance.com:1521:SID";
     // Connect to URL using USERNAME and PASSWORD
     connection = DriverManager.getConnection(url,"USER_NAME",lr.decrypt("password"));
     lr.log_message("JDBC Connection Successful");
        } catch (SQLException e) {
     // If Connection Failed
     lr.log_message("Database Connection Failed, Please check your connection string");
     lr.abort();
 }
     return 0;
    } //end of init

    public int action() throws ClassNotFoundException, SQLException {

 // Database Query Example 1
 lr.start_transaction("Database_Query_1");
 database_query("select BANNER from TABLE_1");
 lr.end_transaction("Database_Query_1", lr.AUTO);

 // Database Query Example 2
 lr.start_transaction("Database_Query_2");
 database_query("SELECT * from TABLE_2");
 lr.end_transaction("Database_Query_2", lr.AUTO);

 // Database Query Example 3
 lr.start_transaction("Database_Query_3");
 database_query("select * from TABLE_3");
 lr.end_transaction("Database_Query_3", lr.AUTO);

 return 0;
    } //end of action

    public int end() throws Throwable {
 connection = null;
        return 0;
    } //end of end

    // Function: database_query
    // Argument: SQL Query String
    // Performs an SQL Query String, and returns pass or fail for the current transaction
    //
    public int database_query(String SQL_QUERY) {
       Statement stmt = null;
       ResultSet rset = null;

       try {
    connection.setAutoCommit(false);
    stmt = connection.createStatement();
    rset = stmt.executeQuery(SQL_QUERY);
    lr.set_transaction_status(lr.PASS);
    // while (rset.next()) {    // Print the results of the query
    //     lr.log_message(rset.getString(1));    // Note: This should be used for debugging only,
    // }       // as it slows down execution time
    //lr.log_message("SQL Query Executed Successfully");
    rset.close();
 } catch (SQLException e) {
     // SQL Query has failed
     lr.log_message("Caught Exception: " + e.getMessage());
     lr.set_transaction_status(lr.FAIL);
     return 1;
 }
 return 0;
    }
}

Performance Monitoring & Analysis on various tiers like Web, Application, Middle Layer and Database levels

Types of Logs which needs to be monitored at various tiers:

Web Server Layer:
  • SSO Logs for Authentication errors
  • Plugin Logs for connectivity error between Web & Application component
  • Access Logs for HTTP Error like HTTP404, HTTP500 etc.
Application Server Layer:
  • sysout logs for exceptions & timeouts
  • syserror logs for database exceptions & application errors
Tibco Layer(Middle Layer):
  • EMS Queues for checking queue depth
  • Mediation Services
  • ESB Operations
Database Layer:


Analyzing Performance Test Results

Once Performance test is completed, the logs and results are collected from the controller machine.

The Performance report can be prepared with following:

LoadRunner Analysis Reports

  • Transaction Response Times
  • Transaction Pass/Failed
  • Throughput (Kilobytes/sec)
  • Errors Rate vs. Load on the system
  • Degradation of response times with load on the system
  • Logs

Performance Testing Types

Smoke or Sanity tests
After Test sets/Load profile created, a ‘Smoke or Sanity tests’ on the scripts will be executed to ensure if the scripts and the application configuration are done correctly. This is must for any kind of performance test. This will be planned for a week. Any defects identified in the dry-run will be fixed during this phase. Smoke test or sanity tests are run with 1 user for each script for the first time. If everything looks good, then actual workload is incorporated to see whether the application is stable under pre-defined load.
If everything goes fine, then actual load and other performance tests like Soak, Failover, Breakdown testing is performed.

Load Tests
The actual test execution happens at UAT Environment. The User load details have to be suggested by the clients or Performance team can decide depending of number of transactions to be achieved (For WEPO Application the normal load is 130 users).
While the test is running, monitors in different tiers (at tool level, application server level, database server level) are observed through Controller and these observations are noted for every test conducted.
Monitoring at application-server level and database-server level will be supported by Client, whenever necessary.

Stress Tests
Stress Testing is normally used to understand the upper limits of capacity within the system. This kind of test is done to determine the system's robustness in terms of extreme load and helps application administrators to determine if the system will perform sufficiently if the current load goes well above the expected maximum.

Soak Tests
Soak Testing, also known as endurance testing, is usually done to determine if the system can sustain the continuous expected load. During soak tests, memory utilization is monitored to detect potential leaks. Also important, but often overlooked is performance degradation, i.e. to ensure that the throughput and/or response times after some long period of sustained activity are as good as or better than at the beginning of the test. It essentially involves applying a significant load to a system for an extended, significant period of time. The goal is to discover how the system behaves under sustained use. It is run for 24 hours for both of the applications.

Failover Tests
After the relevant load, stress and soak tests are performed, failover tests are conducted which determines at what point the application crashed does and this happens in coordination with the production support teams
and database team, which works to first kill the DB node and stop the instance on that DB node for 10 minutes and then restarting the instances. In these 10 minutes time frame it is checked to see whether application crashes completely or it recovers back. If, it recovers it is analyzed how much time it takes for the application to recover back. Also, the number of failed transactions is checked during this time frame to check whether it
is crossing the predefined threshold or it crosses that bar. It was also tested and certified accordingly.

Scalability Tests
In Scalability testing, it is checked that if application is stable under the certified load, then users are subsequently increased and also DB connections was changed to 3500 from 2500 which supports more users load at one point of time and it is tested and certified accordingly. Also, the server’s configuration was increased on the performance testing environment and this was also tested to check that resources are scaled accordingly with good load tests.

Performance Testing Process

Below are the generic Performance Testing Process: