Smoke or Sanity tests
After Test sets/Load profile created, a ‘Smoke or Sanity tests’ on the scripts will be executed to ensure if the scripts and the application configuration are done correctly. This is must for any kind of performance test. This will be planned for a week. Any defects identified in the dry-run will be fixed during this phase. Smoke test or sanity tests are run with 1 user for each script for the first time. If everything looks good, then actual workload is incorporated to see whether the application is stable under pre-defined load.
If everything goes fine, then actual load and other performance tests like Soak, Failover, Breakdown testing is performed.
Load Tests
The actual test execution happens at UAT Environment. The User load details have to be suggested by the clients or Performance team can decide depending of number of transactions to be achieved (For WEPO Application the normal load is 130 users).
While the test is running, monitors in different tiers (at tool level, application server level, database server level) are observed through Controller and these observations are noted for every test conducted.
Monitoring at application-server level and database-server level will be supported by Client, whenever necessary.
Stress Tests
Stress Testing is normally used to understand the upper limits of capacity within the system. This kind of test is done to determine the system's robustness in terms of extreme load and helps application administrators to determine if the system will perform sufficiently if the current load goes well above the expected maximum.
Soak Tests
Soak Testing, also known as endurance testing, is usually done to determine if the system can sustain the continuous expected load. During soak tests, memory utilization is monitored to detect potential leaks. Also important, but often overlooked is performance degradation, i.e. to ensure that the throughput and/or response times after some long period of sustained activity are as good as or better than at the beginning of the test. It essentially involves applying a significant load to a system for an extended, significant period of time. The goal is to discover how the system behaves under sustained use. It is run for 24 hours for both of the applications.
Failover Tests
After the relevant load, stress and soak tests are performed, failover tests are conducted which determines at what point the application crashed does and this happens in coordination with the production support teams
and database team, which works to first kill the DB node and stop the instance on that DB node for 10 minutes and then restarting the instances. In these 10 minutes time frame it is checked to see whether application crashes completely or it recovers back. If, it recovers it is analyzed how much time it takes for the application to recover back. Also, the number of failed transactions is checked during this time frame to check whether it
is crossing the predefined threshold or it crosses that bar. It was also tested and certified accordingly.
Scalability Tests
In Scalability testing, it is checked that if application is stable under the certified load, then users are subsequently increased and also DB connections was changed to 3500 from 2500 which supports more users load at one point of time and it is tested and certified accordingly. Also, the server’s configuration was increased on the performance testing environment and this was also tested to check that resources are scaled accordingly with good load tests.
After Test sets/Load profile created, a ‘Smoke or Sanity tests’ on the scripts will be executed to ensure if the scripts and the application configuration are done correctly. This is must for any kind of performance test. This will be planned for a week. Any defects identified in the dry-run will be fixed during this phase. Smoke test or sanity tests are run with 1 user for each script for the first time. If everything looks good, then actual workload is incorporated to see whether the application is stable under pre-defined load.
If everything goes fine, then actual load and other performance tests like Soak, Failover, Breakdown testing is performed.
Load Tests
The actual test execution happens at UAT Environment. The User load details have to be suggested by the clients or Performance team can decide depending of number of transactions to be achieved (For WEPO Application the normal load is 130 users).
While the test is running, monitors in different tiers (at tool level, application server level, database server level) are observed through Controller and these observations are noted for every test conducted.
Monitoring at application-server level and database-server level will be supported by Client, whenever necessary.
Stress Tests
Stress Testing is normally used to understand the upper limits of capacity within the system. This kind of test is done to determine the system's robustness in terms of extreme load and helps application administrators to determine if the system will perform sufficiently if the current load goes well above the expected maximum.
Soak Tests
Soak Testing, also known as endurance testing, is usually done to determine if the system can sustain the continuous expected load. During soak tests, memory utilization is monitored to detect potential leaks. Also important, but often overlooked is performance degradation, i.e. to ensure that the throughput and/or response times after some long period of sustained activity are as good as or better than at the beginning of the test. It essentially involves applying a significant load to a system for an extended, significant period of time. The goal is to discover how the system behaves under sustained use. It is run for 24 hours for both of the applications.
Failover Tests
After the relevant load, stress and soak tests are performed, failover tests are conducted which determines at what point the application crashed does and this happens in coordination with the production support teams
and database team, which works to first kill the DB node and stop the instance on that DB node for 10 minutes and then restarting the instances. In these 10 minutes time frame it is checked to see whether application crashes completely or it recovers back. If, it recovers it is analyzed how much time it takes for the application to recover back. Also, the number of failed transactions is checked during this time frame to check whether it
is crossing the predefined threshold or it crosses that bar. It was also tested and certified accordingly.
Scalability Tests
In Scalability testing, it is checked that if application is stable under the certified load, then users are subsequently increased and also DB connections was changed to 3500 from 2500 which supports more users load at one point of time and it is tested and certified accordingly. Also, the server’s configuration was increased on the performance testing environment and this was also tested to check that resources are scaled accordingly with good load tests.
No comments:
Post a Comment