Skip to main content

Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "COSMOS Performance Testing"

(→‎Federating CMDB)
(→‎Broker - Multiple Data Managers)
 
(47 intermediate revisions by 3 users not shown)
Line 75: Line 75:
 
|}
 
|}
  
−
== '''Performance - Volume Tests''' ==
+
== '''Test Environment''' ==
  
−
=== Apache JMeter Tool - Setup Instructions ===
+
=== Hardware/Software ===
  
−
#Open Jmeter window (jmeter-install-dir\bin\jmeter.bat)
+
'''Windows PC1'''
−
#On the left side, right click on the ‘Test Plan’ node and select ‘Add->Thread Group’. This inserts one child node called ‘Thread Group’ under ‘Test Plan’.  
+
* Intel-Xeon CPU 3.20GHz
−
#Select the ‘Thread Group’ node. On the right side for ‘Number of Threads’ text box, give the number of samples.  
+
* 3.25GB RAM
−
#Right click on ‘Thread Group’ node, and select ‘Add->Sampler->SOAP/XML- RPC Request’. This inserts one child node called ‘SOAP/XML-RPC Request’ under ‘Thread Group’
+
* Windows 2003 SP2
−
#Select the ‘SOAP/XML-RPC Request’ node. On the right-side, in text box URL: give the URL to which the request to be send. In the ‘Soap/XML-RPC Data’ text area type the ‘SOAP request’
+
* Tomcat 5.5.25 / Axis2
−
#Right click on ‘Thread Group’ node, and select ‘Add->Listener->Summary Report’. This inserts one child node called ‘Summary Report’ under the ‘Thread Group’. We can select our own format for displaying results (summary, table, tree…etc)
+
* COSMOS Build Components
−
#Save the test
+
* JDK 1.5.0_15
−
#Now, to run the test, from menu bar select Run->Start for running a test.  
+
 
−
#Run->Stop for stopping test
+
'''Windows PC2'''
−
#The results will be populated in the Listeners that we have added to the test.  
+
* Intel-Pentium 4 CPU 2.80GHz
 +
* 4GB RAM
 +
* Windows XP Professional + SP2
 +
* Apache JMeter 2.3.2
 +
* JDK 1.5.0_15
 +
 
 +
'''Linux PC1'''
 +
* Intel-Pentium 4 CPU 2.80GHz
 +
* 2GB RAM
 +
* Linux-x86,Kernal-2.6.18-8.el5
 +
* Tomcat 5.5.25 / Axis2
 +
* COSMOS Build Components
 +
* JDK 1.5.0_15
  
 
----
 
----
  
−
=== Broker Testing ===
+
=== Performance Testing Tool ===
 +
 
 +
==== '''Apache JMeter - Introduction'''====
 +
 
 +
Apache JMeter is a tool that can be used to test applications utilizing HTTP or FTP servers. It is Java based and is highly extensible through a provided API. A typical JMeter test involves creating a loop and a thread group. The loop simulates sequential requests to the server with a preset delay. A thread group is designed to simulate a concurrent load. JMeter provides a user interface. It also exposes an API that allows you to run JMeter-based tests from a Java application.
 +
 
 +
To create a load test in JMeter, build a test plan - which is essentially a sequence of operations JMeter will execute.
 +
 
 +
The simplest test plan normally includes the following elements:
 +
* Thread group - These elements are used to specify number of running threads and a ramp-up period. Each thread simulates a user and the ramp-up period specifies the time to create all the threads. For example with 5 threads and 10 seconds of ramp-up time, it will take 2 seconds between each thread creation. The loop count defines the running time for a thread. The scheduler also allows you to set the start and end of the run time.
 +
* Samplers - These elements are configurable requests to the server HTTP, FTP, or LDAP requests. * Listeners - These elements are used to post process request data. For example, you can save data to a file or illustrate the results with a chart.
 +
 
 +
A more detailed description of the available elements is given on the Apache JMeter Web site. In some cases, when the available elements are not suitable for a particular test, a developer can write his or her own script or Java class and embed it into a test plan by placing a jar file into the JMeter installation \lib\ext\ directory.
 +
 
 +
==== '''Apache JMeter - Setup'''====
 +
 
 +
* Download latest binary executable of JMeter from the Web site (http://jakarta.apache.org/jmeter/), unzip it, and the application is ready for use on Windows or Unix platforms
 +
* To use the Web services samples we also need the mail.jar and activation.jar available from Sun Microsystems.After downloading these two jars, place them into the Java classpath or into the JMeter installation lib directory
 +
* Go to JMeter Application's bin folder and start the application with jmeter.bat
 +
* Now, right-click Test Plan and add a Thread Group
 +
* In a tree structure, under Thread Group, add a "WebService (SOAP) Request" and a custom Graph Result viewer (If you cannot add a WebService request to the test plan, you probably don't have either mail.jar or activation.jar in your path.)
 +
* Under Thread Group properties, type in the number of threads, ramp-up periods, and loop count
 +
* Now, we need to set the parameters of the SOAP request sent to the server,manually type in values for server name, port number, path, and SOAP action. As a last step, fill in the SOAP request in the SOAP/XML-RPC Data area.
 +
* After all the fields in the Web service request dialog have been entered, save the JMeter project clicking Ctrl+S
 +
* Now finally run the load test by either clicking Ctrl+R or choosing Run from the menu and clicking Start
 +
 
 +
----
 +
 
 +
==Performance - Volume/Load Tests==
 +
 
 +
* Load testing using JMeter tool with simulation of high load on COSMOS server components and determining its capacity and limitations.
 +
* Tested with number of threads as 100, ramp-up periods as 15ms , and loop count as 10.
 +
 
 +
----
 +
=== Broker ===
  
 
==== Test Broker-getAllGroups ====
 
==== Test Broker-getAllGroups ====
Line 98: Line 144:
 
** Follow the procedure in creating, running the test using JMeter.  
 
** Follow the procedure in creating, running the test using JMeter.  
 
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 +
 
<pre>
 
<pre>
 +
URL: http://localhost:8080/axis2/services/Broker
 +
 
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 
<soapenv:Body><tns:getAllGroups xmlns:tns="http://www.eclipse.org/cosmos/broker" />
 
<soapenv:Body><tns:getAllGroups xmlns:tns="http://www.eclipse.org/cosmos/broker" />
 
</soapenv:Body></soapenv:Envelope>
 
</soapenv:Body></soapenv:Envelope>
 
</pre>
 
</pre>
−
* Result:  
+
* Results:
−
[[Image:Broker_getAllGroups2.png]]
+
 
 +
** i14 Test Result
 +
[[Image:i14_Broker_getAllGroups.png]]
 +
 
 +
** i13 Test Result
 +
[[Image:i13_Broker_getAllGroups.png]]
 +
 
 
----
 
----
  
Line 114: Line 169:
 
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 
<pre>
 
<pre>
 +
URL: http://localhost:8080/axis2/services/StatisticalDataManager
 +
 
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 
<soapenv:Body><tns:getStatDataForSources xmlns:tns="http://datamgr.stat.internal.dc.cosmos.eclipse.org">
 
<soapenv:Body><tns:getStatDataForSources xmlns:tns="http://datamgr.stat.internal.dc.cosmos.eclipse.org">
Line 120: Line 177:
 
</soapenv:Body></soapenv:Envelope>
 
</soapenv:Body></soapenv:Envelope>
 
</pre>
 
</pre>
−
*Result
+
* Results
−
[[Image:StatDataManager_getStatDataSources2.png]]
+
 
 +
** i14 Test Result:
 +
[[Image:i14_StatDataManager_getStatDataForSources.png]]
 +
 
 +
** i13 Test Result:
 +
[[Image:i13_StatDataManager_getStatDataForSources.png]]
 +
 
 
----
 
----
−
=== Example Mdr ===
+
 
−
==== Test ExampleMdr - ExampleMdrQueryService ====
+
=== Logging Data Manager ===
−
* Using Apache JMeter Tool,testing ExampleMdrQueryService with 1000 samples of users simulation  
+
 
 +
==== Test LoggingDataManager - getLogFileIdentifiers ====
 +
* Using Apache JMeter Tool,testing LoggingDataManager with 1000 samples of users simulation  
 
** Follow the procedure in creating, running the test using JMeter.  
 
** Follow the procedure in creating, running the test using JMeter.  
 
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 
<pre>
 
<pre>
 +
URL: http://localhost:8080/axis2/services/CBEServices
 +
 +
<?xml version='1.0' encoding='utf-8'?>
 +
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 +
<soapenv:Body><tns:getLogFileIdentifiers xmlns:tns="http://datamgr.logging.internal.dc.cosmos.eclipse.org" />
 +
</soapenv:Body></soapenv:Envelope>
 +
</pre>
 +
* Results
 +
 +
** i14 Test Result:
 +
[[Image:i14_LogDataManager_getLogFileIdentifiers.png]]
 +
 +
** i13 Test Result:
 +
[[Image:i13_LogDataManager_getLogFileIdentifiers.png]]
 +
 +
----
 +
 +
==== Test LoggingDataManager - getCBEs ====
 +
* Using Apache JMeter Tool,testing LoggingDataManager-getCBEs with 1000 samples of users simulation
 +
** Follow the procedure in creating, running the test using JMeter.
 +
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 +
<pre>
 +
URL: http://localhost:8080/axis2/services/CBEServices
 +
 
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
−
<soapenv:Body><tns:getStatDataForSources xmlns:tns="http://datamgr.stat.internal.dc.cosmos.eclipse.org">
+
<soapenv:Body><tns:getCBEs xmlns:tns="http://datamgr.logging.internal.dc.cosmos.eclipse.org">
−
<tns:dataset>server 33 tomcat1181829221515</tns:dataset><tns:keyset>CBE</tns:keyset><tns:start>0</tns:start>
+
<tns:logfile>access_log.cbe.xml</tns:logfile></tns:getCBEs>
−
<tns:size>1000</tns:size></tns:getStatDataForSources>
+
 
</soapenv:Body></soapenv:Envelope>
 
</soapenv:Body></soapenv:Envelope>
 
</pre>
 
</pre>
−
*Result
+
* Results
−
[[Image:ExampleMdrQueryService2.png]]
+
 
 +
** i14 Test Result:
 +
[[Image:i14_LoggingDataManager_getCBEs.png]]
 +
 
 +
** i13 Test Result:
 +
[[Image:i13_LoggingDataManager_getCBEs.png]]
 +
 
 
----
 
----
−
=== Federating CMDB ===
+
 
 +
=== FederatingCMDB ===
 
==== Test FederatingCMDB - FederatingCMDBRegistrationService ====
 
==== Test FederatingCMDB - FederatingCMDBRegistrationService ====
 
* Using Apache JMeter Tool,testing FederatingCMDBRegistrationService with 1000 samples of users simulation  
 
* Using Apache JMeter Tool,testing FederatingCMDBRegistrationService with 1000 samples of users simulation  
Line 144: Line 239:
 
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 
<pre>
 
<pre>
 +
URL: http://localhost:8080/axis2/services/TestFederatingCMDBRegistrationService
 +
 
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><cmdbf:registerRequest xmlns:cmdbf="http://cmdbf.org/schema/1-0-0/datamodel">
 
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><cmdbf:registerRequest xmlns:cmdbf="http://cmdbf.org/schema/1-0-0/datamodel">
 
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
Line 632: Line 729:
 
</cmdbf:registerRequest></soapenv:Body></soapenv:Envelope>
 
</cmdbf:registerRequest></soapenv:Body></soapenv:Envelope>
 
</pre>
 
</pre>
−
*Result
+
* Results
−
[[Image:TestFederatingCMDBRegistrationService_ExampleMdr.png]]
+
 
 +
** i14 Test Result:
 +
[[Image:i14_FederatingCMDB_Registration.png]]
 +
 
 +
** i13 Test Result:
 +
[[Image:i13_FederatingCMDB_Registration.png]]
 +
 
 +
----
 +
 
 +
==== Test FederatingCMDB - FederatingCMDBQueryService ====
 +
* Using Apache JMeter Tool,testing FederatingCMDBQueryService with 1000 samples of users simulation
 +
** Follow the procedure in creating, running the test using JMeter.
 +
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 +
<pre>
 +
URL: http://localhost:8080/axis2/services/TestFederatingCMDBQueryService
 +
 
 +
<?xml version='1.0' encoding='utf-8'?>
 +
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body>
 +
<s:query xmlns:s="http://cmdbf.org/schema/1-0-0/datamodel">       
 +
<s:itemTemplate id="all-items">        </s:itemTemplate>       
 +
<s:itemTemplate id="all-items-in-relationships">        </s:itemTemplate>        <s:relationshipTemplate id="all-relationships">               
 +
<s:sourceTemplate ref="all-items-in-relationships" />               
 +
<s:targetTemplate ref="all-items-in-relationships" />        </s:relationshipTemplate> </s:query></soapenv:Body></soapenv:Envelope>
 +
</pre>
 +
* Results
 +
 
 +
** i14 Test Result:
 +
[[Image:i14_FederatingCMDB_QueryService.png]]
 +
 
 +
** i13 Test Result:
 +
[[Image:i13_FederatingCMDB_QueryService.png]]
 +
 
 +
----
 +
 
 +
=== Example Mdr ===
 +
==== Test ExampleMdr - ExampleMdrQueryService ====
 +
* Using Apache JMeter Tool,testing ExampleMdrQueryService with 1000 samples of users simulation
 +
** Follow the procedure in creating, running the test using JMeter.
 +
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 +
<pre>
 +
URL: http://localhost:8080/axis2/services/ExampleMdrQueryService
 +
 
 +
<?xml version='1.0' encoding='utf-8'?>
 +
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 +
<soapenv:Body> <st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel">
 +
<st:itemTemplate id="items" suppressFromResult="false"> <st:recordConstraint>
 +
<st:recordType localName="student" namespace="http://schemas.examplemdr.org/schema/ExampleMDR" /> </st:recordConstraint></st:itemTemplate></st:query>
 +
</soapenv:Body> </soapenv:Envelope>
 +
 
 +
</pre>
 +
* Results
 +
 
 +
** i14 Test Result:
 +
[[Image:i14_ExampleMdr_QueryService.png]]
 +
 
 +
** i13 Test Result:
 +
[[Image:i13_ExampleMdr_QueryService.png]]
 +
 
 +
----
 +
 
 +
=== SML MDR ===
 +
==== Test SMLMdr - SMLMdrQueryService ====
 +
* Using Apache JMeter Tool,testing SMLMdrQueryService with 1000 samples of users simulation
 +
** Follow the procedure in creating, running the test using JMeter.
 +
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 +
<pre>
 +
URL: http://localhost:8080/axis2/services/SmlMdrQueryService
 +
 
 +
<?xml version='1.0' encoding='utf-8'?>
 +
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 +
<soapenv:Body>
 +
<s:query xmlns:s="http://cmdbf.org/schema/1-0-0/datamodel">
 +
<s:itemTemplate id="All_Computer_Systems">
 +
<s:recordConstraint> <s:recordType localName="computerSystem" namespace="http://www.cosmos.rm/dataCenter" />
 +
</s:recordConstraint> </s:itemTemplate></s:query>
 +
</soapenv:Body></soapenv:Envelope>
 +
</pre>
 +
* Results
 +
 
 +
** i14 Test Result:
 +
[[Image:i14_SMLMDR_QueryService.png]]
 +
 
 +
** i13 Test Result:
 +
[[Image:i13_SMLMDR_QueryService.png]]
 +
 
 +
----
 +
 
 +
=== AperiMdr ===
 +
==== Test AperiMdr - AperiMdrQueryService ====
 +
* Using Apache JMeter Tool,testing AperiMdrQueryService with 1000 samples of users simulation
 +
** Follow the procedure in creating, running the test using JMeter.
 +
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 +
<pre>
 +
URL: http://localhost:8080/axis2/services/AperiMdrQueryService
 +
 
 +
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><s:query xmlns:s="http://cmdbf.org/schema/1-0-0/datamodel">
 +
  <s:itemTemplate id="items" suppressFromResult="false">
 +
      <s:recordConstraint>
 +
        <s:recordType localName="StorageSystem" namespace="http://schemas.aperimdr.org/schema/AperiMDR">
 +
        </s:recordType>
 +
      </s:recordConstraint>
 +
  </s:itemTemplate>
 +
</s:query></soapenv:Body></soapenv:Envelope>
 +
</pre>
 +
* Results
 +
 
 +
** i14 Test Result:
 +
[[Image:i14_AperiMdr_QueryService.png]]
 +
 
 +
** i13 Test Result:
 +
[[Image:i13_AperiMdr_QueryService.png]]
 +
 
 +
----
 +
 
 +
==Performance - Scalability Tests==
 +
 
 +
* Scalability Tests, with increasing test data under Example MDR
 +
* Scalability Tests, with increasing data managers (6,50,100 datamanagers)
 +
 
 +
=== Example MDR - Volumes of test data ===
 +
* Using Apache JMeter Tool,testing ExampleMDR with multiple volumes of test data (data.xml) 1MB, 2MB,2.5MB with 1000 samples.
 +
** Modify data.xml file (located under CMDBServices.jar file) to insert test data with 2000 student records
 +
** Follow the procedure to create the test, running the test using JMeter.
 +
** Provide below SOAP request of ExampleMDR to Jmeter tool and run it 1000 samples
 +
<pre>
 +
URL: http://localhost:8080/axis2/services/ExampleMdrQueryService
 +
 
 +
<?xml version='1.0' encoding='utf-8'?>
 +
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 +
<soapenv:Body>
 +
<st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel">
 +
<st:itemTemplate id="items" suppressFromResult="false">
 +
<st:recordConstraint> <st:recordType localName="student" namespace="http://schemas.examplemdr.org/schema/ExampleMDR" /> </st:recordConstraint></st:itemTemplate>
 +
</st:query> </soapenv:Body> </soapenv:Envelope>
 +
</pre>
 +
* Results:
 +
 
 +
** i14 Test Result:
 +
[[Image:i14_ExampleMDR_VolumeTest.png]]
 +
 
 +
** i13 Test Result:
 +
[[Image:i13_ExampleMdr_VolumeTest.png]]
 +
 
 +
----
 +
 
 +
=== Broker - Multiple Data Managers ===
 +
* Using Apache JMeter Tool,testing broker with multiple datamanagers (22)
 +
** Create duplicate Data Managers/MDR's by copying and editing some configuration data
 +
** Follow the procedure to create the test, running the test using JMeter.
 +
** Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
 +
<pre>
 +
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 +
<soapenv:Body><tns:getAllGroups xmlns:tns="http://www.eclipse.org/cosmos/broker" />
 +
</soapenv:Body></soapenv:Envelope>
 +
</pre>
 +
*Results:
 +
 
 +
** i14 Test Result:
 +
[[Image:i14_MultipleMdrs_Test.png]]
 +
 
 +
** i13 Test Result:
 +
[[Image:i13_MultipleMdrs_Test.png]]
 +
 
 +
----
 +
 
 +
==References==
 +
 
 +
* Apache JMeter Tool
 +
** http://jakarta.apache.org/jmeter/index.html
 +
* JMeter plugin that provides a new Statistical Aggregate Report listener
 +
** [http://rubenlaguna.com/wp/wp-content/uploads/2007/01/statsaggvisualizereclipseproject.zip Statistical Agreegate Report]
 +
 
 +
----

Latest revision as of 08:25, 10 November 2008

Introduction

Performance testing is an important part of any distributed or Web application testing, which ensures that the application delivered to a customer satisfies high load,availability and scalability requirements.

Early identification of software load limitations helps to configure the system appropriately to avoid unexpected crashes.

Some of the questions which should be addressed at system performance analysis are:

  • Will the Application or Server be able to process simultaneous requests coming from hundreds, or thousands of clients ? and
  • What is the frequency of requests the system can handle ?


Apache JMeter—a performance testing framework from Apache, has been widely accepted as a performance testing tool for Web applications. It can be used to analyze overall server performance under simulated heavy load.

Terminology

The terminologies/acronyms below are commonly used throughout this document. The list below defines each term regarding how it is used in this document.

Term Definition
Scalability Scalability refers to the ability to handle additional workload, without adversely affecting performance, by adding resources such as CPU, memory, and storage capacity.
Throughput Throughput is the number of units of work that can be handled per unit of time; for instance, requests per second, calls per day, hits per second, reports per year, etc.
Median Median is the number that represents the time, where half of server response time is lower than this number and half is higher
Average Average is the total time running divided by number of requests sent to the server.
Deviation Deviation shows how much the server response time varies, a measure of degree of dispersion, or, in other words, how spread the data are
Volume Testing Testing which confirms that any values that may become large over time (such as accumulated counts, logs, and data files), can be accommodated by the program and will not cause the program to stop working or degrade its operation in any manner.
Load Test A performance test focused on determining or validating performance characteristics of the product under test when subjected to workload models and load volumes anticipated during production operations
Performance Test A performance test is a technical investigation done to determine or validate the speed, scalability, and/or stability characteristics of the product under test.
Stress Test A stress test is a type of performance test designed to evaluate an application’s behavior when it is pushed beyond normal or peak load conditions
Response Time Response time is the amount of time that it takes for a server to respond to a request.
Latency In general terms, a time delay between the moment something is initiated, and the moment one of its effects begins. In the realm of software performance, latency is often referred to in the contexts of server latency, network latency and disk latency.
Utilization In the context of performance testing, utilization is the percentage of time that a resource is busy servicing user requests. The remaining percentage of time is considered idle time.

Test Environment

Hardware/Software

Windows PC1

  • Intel-Xeon CPU 3.20GHz
  • 3.25GB RAM
  • Windows 2003 SP2
  • Tomcat 5.5.25 / Axis2
  • COSMOS Build Components
  • JDK 1.5.0_15

Windows PC2

  • Intel-Pentium 4 CPU 2.80GHz
  • 4GB RAM
  • Windows XP Professional + SP2
  • Apache JMeter 2.3.2
  • JDK 1.5.0_15

Linux PC1

  • Intel-Pentium 4 CPU 2.80GHz
  • 2GB RAM
  • Linux-x86,Kernal-2.6.18-8.el5
  • Tomcat 5.5.25 / Axis2
  • COSMOS Build Components
  • JDK 1.5.0_15

Performance Testing Tool

Apache JMeter - Introduction

Apache JMeter is a tool that can be used to test applications utilizing HTTP or FTP servers. It is Java based and is highly extensible through a provided API. A typical JMeter test involves creating a loop and a thread group. The loop simulates sequential requests to the server with a preset delay. A thread group is designed to simulate a concurrent load. JMeter provides a user interface. It also exposes an API that allows you to run JMeter-based tests from a Java application.

To create a load test in JMeter, build a test plan - which is essentially a sequence of operations JMeter will execute.

The simplest test plan normally includes the following elements:

  • Thread group - These elements are used to specify number of running threads and a ramp-up period. Each thread simulates a user and the ramp-up period specifies the time to create all the threads. For example with 5 threads and 10 seconds of ramp-up time, it will take 2 seconds between each thread creation. The loop count defines the running time for a thread. The scheduler also allows you to set the start and end of the run time.
  • Samplers - These elements are configurable requests to the server HTTP, FTP, or LDAP requests. * Listeners - These elements are used to post process request data. For example, you can save data to a file or illustrate the results with a chart.

A more detailed description of the available elements is given on the Apache JMeter Web site. In some cases, when the available elements are not suitable for a particular test, a developer can write his or her own script or Java class and embed it into a test plan by placing a jar file into the JMeter installation \lib\ext\ directory.

Apache JMeter - Setup

  • Download latest binary executable of JMeter from the Web site (http://jakarta.apache.org/jmeter/), unzip it, and the application is ready for use on Windows or Unix platforms
  • To use the Web services samples we also need the mail.jar and activation.jar available from Sun Microsystems.After downloading these two jars, place them into the Java classpath or into the JMeter installation lib directory
  • Go to JMeter Application's bin folder and start the application with jmeter.bat
  • Now, right-click Test Plan and add a Thread Group
  • In a tree structure, under Thread Group, add a "WebService (SOAP) Request" and a custom Graph Result viewer (If you cannot add a WebService request to the test plan, you probably don't have either mail.jar or activation.jar in your path.)
  • Under Thread Group properties, type in the number of threads, ramp-up periods, and loop count
  • Now, we need to set the parameters of the SOAP request sent to the server,manually type in values for server name, port number, path, and SOAP action. As a last step, fill in the SOAP request in the SOAP/XML-RPC Data area.
  • After all the fields in the Web service request dialog have been entered, save the JMeter project clicking Ctrl+S
  • Now finally run the load test by either clicking Ctrl+R or choosing Run from the menu and clicking Start

Performance - Volume/Load Tests

  • Load testing using JMeter tool with simulation of high load on COSMOS server components and determining its capacity and limitations.
  • Tested with number of threads as 100, ramp-up periods as 15ms , and loop count as 10.

Broker

Test Broker-getAllGroups

  • Using Apache JMeter Tool,testing Broker with 1000 samples of users simulation
    • Follow the procedure in creating, running the test using JMeter.
    • Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
URL: http://localhost:8080/axis2/services/Broker

<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body><tns:getAllGroups xmlns:tns="http://www.eclipse.org/cosmos/broker" />
</soapenv:Body></soapenv:Envelope>
  • Results:
    • i14 Test Result

I14 Broker getAllGroups.png

    • i13 Test Result

I13 Broker getAllGroups.png

Statistical Data Manager

Test StatisticalDataManager - getStatDataForSources

  • Using Apache JMeter Tool,testing StatisticalDataManager with 1000 samples of users simulation
    • Follow the procedure in creating, running the test using JMeter.
    • Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
URL: http://localhost:8080/axis2/services/StatisticalDataManager

<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body><tns:getStatDataForSources xmlns:tns="http://datamgr.stat.internal.dc.cosmos.eclipse.org">
<tns:dataset>server 33 tomcat1181829221515</tns:dataset><tns:keyset>CBE</tns:keyset><tns:start>0</tns:start>
<tns:size>1000</tns:size></tns:getStatDataForSources>
</soapenv:Body></soapenv:Envelope>
  • Results
    • i14 Test Result:

I14 StatDataManager getStatDataForSources.png

    • i13 Test Result:

I13 StatDataManager getStatDataForSources.png

Logging Data Manager

Test LoggingDataManager - getLogFileIdentifiers

  • Using Apache JMeter Tool,testing LoggingDataManager with 1000 samples of users simulation
    • Follow the procedure in creating, running the test using JMeter.
    • Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
URL: http://localhost:8080/axis2/services/CBEServices

<?xml version='1.0' encoding='utf-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body><tns:getLogFileIdentifiers xmlns:tns="http://datamgr.logging.internal.dc.cosmos.eclipse.org" />
</soapenv:Body></soapenv:Envelope>
  • Results
    • i14 Test Result:

I14 LogDataManager getLogFileIdentifiers.png

    • i13 Test Result:

I13 LogDataManager getLogFileIdentifiers.png

Test LoggingDataManager - getCBEs

  • Using Apache JMeter Tool,testing LoggingDataManager-getCBEs with 1000 samples of users simulation
    • Follow the procedure in creating, running the test using JMeter.
    • Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
URL: http://localhost:8080/axis2/services/CBEServices

<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body><tns:getCBEs xmlns:tns="http://datamgr.logging.internal.dc.cosmos.eclipse.org">
<tns:logfile>access_log.cbe.xml</tns:logfile></tns:getCBEs>
</soapenv:Body></soapenv:Envelope>
  • Results
    • i14 Test Result:

I14 LoggingDataManager getCBEs.png

    • i13 Test Result:

I13 LoggingDataManager getCBEs.png

FederatingCMDB

Test FederatingCMDB - FederatingCMDBRegistrationService

  • Using Apache JMeter Tool,testing FederatingCMDBRegistrationService with 1000 samples of users simulation
    • Follow the procedure in creating, running the test using JMeter.
    • Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
URL: http://localhost:8080/axis2/services/TestFederatingCMDBRegistrationService

<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><cmdbf:registerRequest xmlns:cmdbf="http://cmdbf.org/schema/1-0-0/datamodel">
	<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
	<cmdbf:itemList>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<student>
									
									<identity firstName="Mike" lastName="Lee" id="Student03">
									</identity>
									
									</student>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Mike Lee</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student03</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<student>
									
									<identity firstName="Jane" lastName="Ryerson" id="Student02">
									</identity>
									
									</student>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Jane Ryerson</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student02</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<student>
									
									<identity firstName="Bob" lastName="Davidson" id="Student01">
									</identity>
									
									</student>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Bob Davidson</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<teacher>
									
									<identity firstName="Heather" lastName="Reeba" id="Staff02">
									</identity>
									
									</teacher>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Heather Reeba</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff02</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<teacher>
									
									<identity firstName="Dawn" lastName="Johnson" id="Staff01">
									</identity>
									
									</teacher>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Dawn Johnson</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<class name="Economics" courseCode="ECM01">
									
									<students>
									
									<enrolledStudent idRef="Student01">
									</enrolledStudent>
									
									<enrolledStudent idRef="Student03">
									</enrolledStudent>
									
									</students>
									
									<teacher idRef="Staff01">
									</teacher>
									
									</class>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Economics</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>ECM01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<class name="Mathematics" courseCode="MAT01">
									
									<students>
									
									<enrolledStudent idRef="Student01">
									</enrolledStudent>
									
									<enrolledStudent idRef="Student02">
									</enrolledStudent>
									
									</students>
									
									<teacher idRef="Staff02">
									</teacher>
									
									</class>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Mathematics</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>MAT01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<class name="Physics" courseCode="PHY01">
									
									<students>
									
									<enrolledStudent idRef="Student02">
									</enrolledStudent>
									
									<enrolledStudent idRef="Student03">
									</enrolledStudent>
									
									</students>
									
									<teacher idRef="Staff02">
									</teacher>
									
									</class>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Physics</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>PHY01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<student>
									
									<identity firstName="Mike" lastName="Lee" id="Student03">
									</identity>
									
									</student>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Mike Lee</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student03</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<student>
									
									<identity firstName="Jane" lastName="Ryerson" id="Student02">
									</identity>
									
									</student>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Jane Ryerson</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student02</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<student>
									
									<identity firstName="Bob" lastName="Davidson" id="Student01">
									</identity>
									
									</student>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Bob Davidson</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<teacher>
									
									<identity firstName="Heather" lastName="Reeba" id="Staff02">
									</identity>
									
									</teacher>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Heather Reeba</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff02</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
		<cmdbf:item>
			<cmdbf:record xmlns="http://school">
				
					<teacher>
									
									<identity firstName="Dawn" lastName="Johnson" id="Staff01">
									</identity>
									
									</teacher>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Dawn Johnson</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:item>
	</cmdbf:itemList>
	<cmdbf:relationshipList>
		<cmdbf:relationship>
			<cmdbf:source>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff02</cmdbf:localId>
			</cmdbf:source>
			<cmdbf:target>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student03</cmdbf:localId>
			</cmdbf:target>
			<cmdbf:record xmlns="http://school">
				
					<class name="Physics" courseCode="PHY01">
									
									<students>
									
									<enrolledStudent idRef="Student02">
									</enrolledStudent>
									
									<enrolledStudent idRef="Student03">
									</enrolledStudent>
									
									</students>
									
									<teacher idRef="Staff02">
									</teacher>
									
									</class>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Physics</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff02/Student03/PHY01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:relationship>
		<cmdbf:relationship>
			<cmdbf:source>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff02</cmdbf:localId>
			</cmdbf:source>
			<cmdbf:target>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student02</cmdbf:localId>
			</cmdbf:target>
			<cmdbf:record xmlns="http://school">
				
					<class name="Mathematics" courseCode="MAT01">
									
									<students>
									
									<enrolledStudent idRef="Student01">
									</enrolledStudent>
									
									<enrolledStudent idRef="Student02">
									</enrolledStudent>
									
									</students>
									
									<teacher idRef="Staff02">
									</teacher>
									
									</class>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Mathematics</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff02/Student02/MAT01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:relationship>
		<cmdbf:relationship>
			<cmdbf:source>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff02</cmdbf:localId>
			</cmdbf:source>
			<cmdbf:target>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student02</cmdbf:localId>
			</cmdbf:target>
			<cmdbf:record xmlns="http://school">
				
					<class name="Physics" courseCode="PHY01">
									
									<students>
									
									<enrolledStudent idRef="Student02">
									</enrolledStudent>
									
									<enrolledStudent idRef="Student03">
									</enrolledStudent>
									
									</students>
									
									<teacher idRef="Staff02">
									</teacher>
									
									</class>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Physics</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff02/Student02/PHY01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:relationship>
		<cmdbf:relationship>
			<cmdbf:source>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff02</cmdbf:localId>
			</cmdbf:source>
			<cmdbf:target>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student01</cmdbf:localId>
			</cmdbf:target>
			<cmdbf:record xmlns="http://school">
				
					<class name="Mathematics" courseCode="MAT01">
									
									<students>
									
									<enrolledStudent idRef="Student01">
									</enrolledStudent>
									
									<enrolledStudent idRef="Student02">
									</enrolledStudent>
									
									</students>
									
									<teacher idRef="Staff02">
									</teacher>
									
									</class>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Mathematics</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff02/Student01/MAT01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:relationship>
		<cmdbf:relationship>
			<cmdbf:source>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff01</cmdbf:localId>
			</cmdbf:source>
			<cmdbf:target>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student03</cmdbf:localId>
			</cmdbf:target>
			<cmdbf:record xmlns="http://school">
				
					<class name="Economics" courseCode="ECM01">
									
									<students>
									
									<enrolledStudent idRef="Student01">
									</enrolledStudent>
									
									<enrolledStudent idRef="Student03">
									</enrolledStudent>
									
									</students>
									
									<teacher idRef="Staff01">
									</teacher>
									
									</class>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Economics</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff01/Student03/ECM01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:relationship>
		<cmdbf:relationship>
			<cmdbf:source>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff01</cmdbf:localId>
			</cmdbf:source>
			<cmdbf:target>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Student01</cmdbf:localId>
			</cmdbf:target>
			<cmdbf:record xmlns="http://school">
				
					<class name="Economics" courseCode="ECM01">
									
									<students>
									
									<enrolledStudent idRef="Student01">
									</enrolledStudent>
									
									<enrolledStudent idRef="Student03">
									</enrolledStudent>
									
									</students>
									
									<teacher idRef="Staff01">
									</teacher>
									
									</class>
				<cmdbf:recordMetadata>
					<cmdbf:recordId>Record for Economics</cmdbf:recordId>
				</cmdbf:recordMetadata>
			</cmdbf:record>
			<cmdbf:instanceId>
				<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
				<cmdbf:localId>Staff01/Student01/ECM01</cmdbf:localId>
			</cmdbf:instanceId>
		</cmdbf:relationship>
	</cmdbf:relationshipList>
</cmdbf:registerRequest></soapenv:Body></soapenv:Envelope>
  • Results
    • i14 Test Result:

I14 FederatingCMDB Registration.png

    • i13 Test Result:

I13 FederatingCMDB Registration.png

Test FederatingCMDB - FederatingCMDBQueryService

  • Using Apache JMeter Tool,testing FederatingCMDBQueryService with 1000 samples of users simulation
    • Follow the procedure in creating, running the test using JMeter.
    • Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
URL: http://localhost:8080/axis2/services/TestFederatingCMDBQueryService

<?xml version='1.0' encoding='utf-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body>
<s:query xmlns:s="http://cmdbf.org/schema/1-0-0/datamodel">         
<s:itemTemplate id="all-items">         </s:itemTemplate>         
<s:itemTemplate id="all-items-in-relationships">         </s:itemTemplate>         <s:relationshipTemplate id="all-relationships">                 
<s:sourceTemplate ref="all-items-in-relationships" />                 
<s:targetTemplate ref="all-items-in-relationships" />         </s:relationshipTemplate> </s:query></soapenv:Body></soapenv:Envelope>
  • Results
    • i14 Test Result:

I14 FederatingCMDB QueryService.png

    • i13 Test Result:

I13 FederatingCMDB QueryService.png

Example Mdr

Test ExampleMdr - ExampleMdrQueryService

  • Using Apache JMeter Tool,testing ExampleMdrQueryService with 1000 samples of users simulation
    • Follow the procedure in creating, running the test using JMeter.
    • Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
URL: http://localhost:8080/axis2/services/ExampleMdrQueryService

<?xml version='1.0' encoding='utf-8'?> 
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> 
<soapenv:Body> <st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel"> 
<st:itemTemplate id="items" suppressFromResult="false"> <st:recordConstraint> 
<st:recordType localName="student" namespace="http://schemas.examplemdr.org/schema/ExampleMDR" /> </st:recordConstraint></st:itemTemplate></st:query> 
</soapenv:Body> </soapenv:Envelope>
  • Results
    • i14 Test Result:

I14 ExampleMdr QueryService.png

    • i13 Test Result:

I13 ExampleMdr QueryService.png

SML MDR

Test SMLMdr - SMLMdrQueryService

  • Using Apache JMeter Tool,testing SMLMdrQueryService with 1000 samples of users simulation
    • Follow the procedure in creating, running the test using JMeter.
    • Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
URL: http://localhost:8080/axis2/services/SmlMdrQueryService

<?xml version='1.0' encoding='utf-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body>
<s:query xmlns:s="http://cmdbf.org/schema/1-0-0/datamodel">	
<s:itemTemplate id="All_Computer_Systems">		
<s:recordConstraint>			<s:recordType localName="computerSystem" namespace="http://www.cosmos.rm/dataCenter" />		
</s:recordConstraint>	</s:itemTemplate></s:query>
</soapenv:Body></soapenv:Envelope>
  • Results
    • i14 Test Result:

I14 SMLMDR QueryService.png

    • i13 Test Result:

I13 SMLMDR QueryService.png

AperiMdr

Test AperiMdr - AperiMdrQueryService

  • Using Apache JMeter Tool,testing AperiMdrQueryService with 1000 samples of users simulation
    • Follow the procedure in creating, running the test using JMeter.
    • Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
URL: http://localhost:8080/axis2/services/AperiMdrQueryService

<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><s:query xmlns:s="http://cmdbf.org/schema/1-0-0/datamodel">
   <s:itemTemplate id="items" suppressFromResult="false">
      <s:recordConstraint>
         <s:recordType localName="StorageSystem" namespace="http://schemas.aperimdr.org/schema/AperiMDR">
         </s:recordType>
      </s:recordConstraint>
   </s:itemTemplate>
</s:query></soapenv:Body></soapenv:Envelope>
  • Results
    • i14 Test Result:

I14 AperiMdr QueryService.png

    • i13 Test Result:

I13 AperiMdr QueryService.png

Performance - Scalability Tests

  • Scalability Tests, with increasing test data under Example MDR
  • Scalability Tests, with increasing data managers (6,50,100 datamanagers)

Example MDR - Volumes of test data

  • Using Apache JMeter Tool,testing ExampleMDR with multiple volumes of test data (data.xml) 1MB, 2MB,2.5MB with 1000 samples.
    • Modify data.xml file (located under CMDBServices.jar file) to insert test data with 2000 student records
    • Follow the procedure to create the test, running the test using JMeter.
    • Provide below SOAP request of ExampleMDR to Jmeter tool and run it 1000 samples
URL: http://localhost:8080/axis2/services/ExampleMdrQueryService

<?xml version='1.0' encoding='utf-8'?> 
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> 
<soapenv:Body> 
<st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel"> 
<st:itemTemplate id="items" suppressFromResult="false"> 
<st:recordConstraint> <st:recordType localName="student" namespace="http://schemas.examplemdr.org/schema/ExampleMDR" /> </st:recordConstraint></st:itemTemplate>
</st:query> </soapenv:Body> </soapenv:Envelope>
  • Results:
    • i14 Test Result:

I14 ExampleMDR VolumeTest.png

    • i13 Test Result:

I13 ExampleMdr VolumeTest.png

Broker - Multiple Data Managers

  • Using Apache JMeter Tool,testing broker with multiple datamanagers (22)
    • Create duplicate Data Managers/MDR's by copying and editing some configuration data
    • Follow the procedure to create the test, running the test using JMeter.
    • Provide below SOAP request of Broker to Jmeter tool and run it with 1000 samples
<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Body><tns:getAllGroups xmlns:tns="http://www.eclipse.org/cosmos/broker" />
</soapenv:Body></soapenv:Envelope>
  • Results:
    • i14 Test Result:

I14 MultipleMdrs Test.png

    • i13 Test Result:

I13 MultipleMdrs Test.png

References

Retrieved from "https://wiki.eclipse.org/index.php?title=COSMOS_Performance_Testing&oldid=127785"

This page was last modified 08:25, 10 November 2008 by SrinivasReddy. Based on work by Srinivas Reddy and Raghuveer Reddy Bathini.

Back to the top