Skip to main content

Notice: this Wiki will be going read only early in 2024 and edits will no longer be possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.

Jump to: navigation, search

Difference between revisions of "COSMOS i10 QA Phase Execution"

(→‎E2E testing status (WinXP SP2/Win2K3 SP1/RHES 5.0))
(→‎QA Test Results)
 
(42 intermediate revisions by the same user not shown)
Line 15: Line 15:
  
 
==QA Status Summary ==
 
==QA Status Summary ==
 +
 +
'''Iteration 10 QA testing effort and completion status (21 Apr’08 – 02 May’08)'''
 +
*QA announces the finish of our testing effort for COSMOS iteration 10 with 1 day ahead of scheduled completion date (02 May, 2008)
 +
*The following activities were completed by QA, as per defined scope of i10 QA activities (http://wiki.eclipse.org/COSMOS_QA_i10_Activities)
 +
 +
'''i10 QA Execution Summary:'''
 +
*Build ID: COSMOS-1.0.0-200804270312 (http://www.eclipse.org/cosmos/downloads/?buildId=COSMOS-1.0.0-200804270312)
 +
#End2End testing on 3 platforms (Windows XP,2003 and Red Hat Enterprise Server 5.0) based on i10 demo page instructions (http://wiki.eclipse.org/COSMOS_DEMO_i10)
 +
#Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite and DataVisualization.UI.Reports.testsuite)
 +
#Review of i10 ER’s for JUnits and Manual Tests
 +
#Additional E2E tests execution (Negative tests) (http://wiki.eclipse.org/COSMOS_i10_QA_Phase_Execution#i10_End2End_Supplement_Tests_.28Negative.29)
 +
#Limited Performance Tests (Configuration tests & Concurrency tests)
 +
 +
'''i10 QA Execution Status Page:'''
 +
*For all above activities, complete status has been captured on http://wiki.eclipse.org/COSMOS_i10_QA_Phase_Execution
 +
 +
'''i10 bugs list filed by QA:'''
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=228157
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=228834
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=228158
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=228374
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=221118
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=229109
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=229277
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=229102
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=229593
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=229599
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=229105
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=229600
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=228159
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=228161
 +
*https://bugs.eclipse.org/bugs/show_bug.cgi?id=229591
  
 
==E2E testing status (WinXP SP2/Win2K3 SP1/RHES 5.0) ==
 
==E2E testing status (WinXP SP2/Win2K3 SP1/RHES 5.0) ==
Line 32: Line 64:
 
|-
 
|-
 
|<cosmos-demo>/bin/configDemo.bat <tomcatDir> <cosmosDir> <birt-path> <dojo-path> <axis2war-path>
 
|<cosmos-demo>/bin/configDemo.bat <tomcatDir> <cosmosDir> <birt-path> <dojo-path> <axis2war-path>
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 
|None
 
|None
 
|}
 
|}
Line 52: Line 84:
 
|-
 
|-
 
|<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:8080/axis2/services/Broker
 
|<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:8080/axis2/services/Broker
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 
|None
 
|None
 
|-
 
|-
−
|broker registerAllDataManagers
+
|COSMOS> broker registerAllDataManagers
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 +
|Found warnings - "log4j:WARN No appenders could be found for logger (org.apache.axis2.util.Loader).
 +
log4j:WARN Please initialize the log4j system properly."
 +
|-
 +
|COSMOS> broker getDataManagers
 +
|Passed
 +
|Passed
 +
|Passed
 
|None
 
|None
 
|-
 
|-
−
|broker getDataManagers
+
|COSMOS> finddatamanagers http://localhost:8080/axis2/services/CosmosServiceFinder
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 
|None
 
|None
 
|-
 
|-
−
|finddatamanagers
+
|COSMOS> broker deregister domsr01-xp StatDataManager
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 
|None
 
|None
 
|-
 
|-
−
|broker deregister HOST StatDataManager
+
|COSMOS> broker deregister domsr01-xp LoggingDataManager
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 
|None
 
|None
 
|-
 
|-
−
|broker deregister HOST LoggingDataManager
+
|COSMOS> broker deregister domsr01-xp TestFederatingCMDB
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 
|None
 
|None
 
|-
 
|-
−
|broker deregister HOST TestFederatingCMDB
+
|COSMOS> broker deregister domsr01-xp ExampleMdr
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 
|None
 
|None
 
|-
 
|-
−
|broker deregister HOST ExampleMdr
+
|COSMOS> broker deregister domsr01-xp SMLMDR
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 
|None
 
|None
 
|-
 
|-
−
|broker deregister HOST SMLMDR
+
|COSMOS> broker deregister domsr01-xp AperiMdr
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 
|None
 
|None
 
|-
 
|-
−
|broker deregister HOST AperiMdr
+
|COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt
−
|
+
|Passed
−
|
+
|Passed
−
|
+
|Passed
 
|None
 
|None
−
}
+
|-
 +
|COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testPropertyQuery.txt
 +
|Passed
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-students.txt
 +
|Passed
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-teachers.txt
 +
|Passed
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\relationship-all-classes.txt
 +
|Passed
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\teaches-relationship.txt
 +
|Passed
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt
 +
|Passed
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_localid.txt
 +
|Passed
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_display_name.txt
 +
|Passed
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_relationship.txt
 +
|Passed
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|COSMOS> COSMOS> graphQuery domsr01-xp TestFederatingCMDB D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\TestFederatingCMDB\all_items.txt
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Empty Response, no data found.
 +
|}
  
−
==i10 ERs - JUnits/Manual Tests Inspection ==
+
'''Using the COSMOS Web User Interface'''
  
 
{|{{BMTableStyle}}
 
{|{{BMTableStyle}}
 
|-{{BMTHStyle}}
 
|-{{BMTHStyle}}
−
! Bugzilla ID
+
! Test Description
−
! ID Summary
+
! Test Status
−
! JUnits Status
+
  Windows XP SP2
−
! Manual Tests 
+
! Test Status
 +
  Windows 2003 SP1
 +
! Test Status
 +
  RHES 5.0
 
! Remarks
 
! Remarks
 
|-
 
|-
 +
|http://localhost:8080/COSMOSUI/?page=cosmos
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Found All 6 DataManagers under COSMOS Web UI
 +
|-
 +
|StatDataManager
 +
server 33 tomcat1181829221515 CBE
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Statistical Report generated successfully
 +
|-
 +
|StatDataManager
 +
server 33 tomcat1181829221515 TomcatSet
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Statistical Report generated successfully
 +
|-
 +
|LoggingDataManager
 +
access_log.cbe.xml
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Top 10 Log Report generated successfully
 +
|-
 +
|LoggingDataManager
 +
windows_app_log.cbe.xml
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Top 10 Log Report generated successfully
 +
|-
 +
|Sample Repository with Registration
 +
Submit Query
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Query for all-items resulted Empty Reponse, no data found.
 +
|-
 +
|Sample Repository with Registration
 +
Register Configuration Items...
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Registration with Example MDR is successful
 +
|-
 +
|Sample Repository with Registration
 +
Deregister Configuration Items...
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Deregistration with Example MDR is successful
 +
|-
 +
|Sample Repository with Registration
 +
Submit Query
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Query for all-items resulted Empty Reponse, no data found.
 +
|-
 +
|Sample Repository with Registration
 +
View MetaData
 +
|Passed
 +
|Passed
 +
|Passed
 +
|No Metadata available
 +
|-
 +
|Resource Repository
 +
Submit Query
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Queries executed successfully with proper results
 +
|-
 +
|Resource Repository
 +
View MetaData
 +
|Passed
 +
|Passed
 +
|Passed
 +
|No Metadata available
 +
|-
 +
|AperiMDR
 +
Submit Query
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Using Query builder able to execute and validate queries properly
 +
|-
 +
|AperiMDR
 +
View MetaData
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Displayed MetaData properly
 +
|-
 +
|Sample Repository with Query
 +
Get All Students
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Displayed Query Result for Students data correctly
 +
|-
 +
|Sample Repository with Query
 +
Get All Teachers
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Displayed Query Result for Teachers data correctly
 +
|-
 +
|Sample Repository with Query
 +
Get All Classes
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Displayed Query Result for Classes data correctly
 +
|-
 +
|Sample Repository with Query
 +
Staff01
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Displayed data for 'students taught by teacher-staff01' correctly
 +
|-
 +
|Sample Repository with Query
 +
Staff02
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Displayed data for 'students taught by teacher-staff02' correctly
 +
|-
 +
|Sample Repository with Query
 +
Submit Query
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Using Query Builder, able to create and execute queries correctly
 +
|-
 +
|Sample Repository with Query
 +
View MetaData
 +
|Passed
 +
|Passed
 +
|Passed
 +
|Displayed MetaData information correctly
 +
|}
  
 +
==Data Visualization TPTP Manual Tests ==
 +
 +
'''/org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite'''
 +
 +
{|{{BMTableStyle}}
 +
|-{{BMTHStyle}}
 +
! Test Description
 +
! Test Status
 +
  Windows XP SP2
 +
! Test Status
 +
  RHES 5.0
 +
! Remarks
 +
|-
 +
|navigate
 +
|Passed
 +
|Passed
 +
|6 nodes should be shown under "Data Managers"
 +
No "Monitoring Data Repository" node, it should be "StatDataManager" node
 +
|-
 +
|navigate2
 +
|Passed
 +
|Passed
 +
|No "Create Query" found, it should be "Submit Query"
 +
|-
 +
|properties
 +
|Passed
 +
|Passed
 +
|Given EPR is wrong, it should be http://localhost:8080/axis2/services/ExampleMdrDataManager
 +
|-
 +
|visualize log report
 +
|Passed
 +
|Passed
 +
|No "Monitoring Data Repository" available, it should be "StatDataManager
 +
|-
 +
|visualize stat report
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|properties 2
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|example cmdbf query
 +
|Passed
 +
|Passed
 +
|XML output contents seems to be wrong, actual result shows only one student detail - "firstName="Mike" id="03 lastname="Lee""
 +
|-
 +
|SML CMDBF Graph Response Viewer
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|Example MDR CMDBF Graph Response Viewer
 +
|Passed
 +
|Passed
 +
|No cancel button available under details view window, it should be OK button.
 +
|-
 +
|Multiple Graph Response Viewers
 +
|Passed
 +
|Passed
 +
|No "Cancel" button available under detail area, it should be Ok button
 +
|-
 +
|Registration Example
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|DeRegistration Example
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|Partial Registration of MDR with CMDBf
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|Partial DeRegistration of MDR with CMDBf
 +
|Passed
 +
|Passed
 +
|None
 +
|}
 +
 +
'''/org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Reports.testsuite'''
 +
 +
{|{{BMTableStyle}}
 +
|-{{BMTHStyle}}
 +
! Test Description
 +
! Test Status
 +
  Windows XP SP2
 +
! Test Status
 +
  RHES 5.0
 +
! Remarks
 +
|-
 +
|LogReport
 +
|Passed
 +
|Passed
 +
|None
 +
|-
 +
|StatReport
 +
|Passed
 +
|Passed
 +
|None
 
|}
 
|}
  
 
==i10 End2End Supplement Tests (Negative) ==
 
==i10 End2End Supplement Tests (Negative) ==
 +
 +
'''COSMOS Command Line Client : Management Console'''
  
 
{|{{BMTableStyle}}
 
{|{{BMTableStyle}}
Line 136: Line 491:
 
! Remarks
 
! Remarks
 
|-
 
|-
 +
|Configure tomcat port to 9080 (other than default port : 8080) and test the following:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
 +
Expected Result:
 +
#Should display COSMOS prompt without any errors
 +
|Passed
 +
|Passed
 +
|None
 +
 +
|-
 +
|Test Broker registration other than defaul port 8080 (say 9080):
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder
 +
Expected Result:
 +
#All DataManagers should successfuly get registered to broker on port 9080
 +
|Failed
 +
|Failed
 +
|DataManagers are getting registered on default port 8080 only, ignored actual port 9080
 +
 +
|-
 +
|Test Broker registration multiple times:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder
 +
#COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder
 +
Expected Result:
 +
#Under tomcat console window, it should display "Service has already been registered at this broker.  No new brokers are register
 +
ed."
 +
|Passed
 +
|Passed
 +
|Found expected message under tomcat console window
 +
 +
|-
 +
|Without broker registration, test the following:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> broker getDataManagers
 +
Expected Result:
 +
#Should display empty list of DataManagers
 +
|Passed
 +
|Passed
 +
|None
 +
 +
|-
 +
|After succesful broker registration, test the following:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> broker getDataManagers
 +
Expected Result:
 +
#Should display all 6 DataManagers registered with broker
 +
|Passed
 +
|Passed
 +
|None
 +
 +
|-
 +
|Without any broker registration, test the following
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> finddatamanagers http://localhost:9080/axis2/services/CosmosServiceFinder
 +
Expected Result:
 +
#Should display all 6 DataManagers deployed under tomcat with port 9080
 +
|Passed
 +
|Passed
 +
|None
 +
 +
|-
 +
|Test deregistration of StatDataManager with valid hostname and valid datamanager name :
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> broker deregister domsr01-xp StatDataManager
 +
Expected Result:
 +
#Deregistration should be successful
 +
|Passed
 +
|Passed
 +
|None
 +
 +
|-
 +
|Test deregistration of StatDataManager with invalid hostname and valid datamanager name :
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> broker deregister unknown StatDataManager
 +
Expected Result:
 +
#Deregistration should fail and should display appropriate error message
 +
|Failed
 +
|Failed
 +
|No Failure message, it still shows "DataManager deregistered"
 +
 +
|-
 +
|Test deregistration of StatDataManager with valid hostname and invalid datamanager name :
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> broker deregister domsr01-xp StatDataMgr
 +
Expected Result:
 +
#Deregistration should fail and should display appropriate error message
 +
|Failed
 +
|Failed
 +
|No Failure message, it still shows "DataManager deregistered"
 +
 +
|-
 +
|Test graphQuery for SMLMDR, with valid hostname,valid DataManager,valid query path:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt
 +
Expected Result:
 +
#Query execution should be succesful.
 +
|Passed
 +
|Passed
 +
|This is depended on Broker registration issue, fixed manually to test this.
 +
 +
|-
 +
|Test graphQuery for SMLMDR, with invalid hostname,valid DataManager,valid query path:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> graphQuery unknown SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt
 +
Expected Result:
 +
#Query execution should fail and throw appropriate error message.
 +
|Passed
 +
|Passed
 +
|Displaying error message as "SMLMDR is not registered at the broker."
 +
 +
|-
 +
|Test graphQuery for SMLMDR, with valid hostname,invalid DataManager,valid query path:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> graphQuery domsr01-xp StatDataManager D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt
 +
Expected Result:
 +
#Query execution should fail and throw appropriate error message.
 +
|Passed
 +
|Passed
 +
|Displayed error message as "StatDataManager is not registered at the broker."
 +
 +
|-
 +
|Test graphQuery for SMLMDR, with valid hostname,valid DataManager,invalid query path:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDRTest\testInstanceIdQuery.txt
 +
Expected Result:
 +
#Query execution should fail and throw appropriate error message.
 +
|Passed
 +
|Passed
 +
|Displayed "java.io.FileNotFoundException:" error message tree
 +
 +
|-
 +
|Test graphQuery for SMLMDR, with valid hostname,valid DataManager,valid query path with spaces:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> graphQuery domsr01-xp SMLMDR "D:\cosmos i10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt"
 +
Expected Result:
 +
#Query execution should be successful
 +
|Failed
 +
|Failed
 +
|Displayed "java.io.FileNotFoundException:" error message tree
 +
 +
|-
 +
|Test graphQuery for AperMdr with valid data, with valid Aperi database settings:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt
 +
Expected Result:
 +
#Query should execute successfully
 +
|Passed
 +
|Passed
 +
|Query executed successfully
 +
 +
|-
 +
|Test graphQuery for AperMdr with invalid query data, with valid Aperi database settings:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt
 +
Expected Result:
 +
#Query execution should fail and display appropriate error message
 +
|Passed
 +
|Passed
 +
|Query execution failed with appropriate error message
 +
 +
|-
 +
|Test graphQuery for AperMdr with valid data, with invalid Aperi database settings:
 +
#<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
 +
#COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt
 +
Expected Result:
 +
#Query execution should fail, with DB Connection error
 +
|Passed
 +
|Passed
 +
|Displayed "aperiDB.getStorageSystems error"
 +
|}
 +
 +
'''Using the COSMOS Web User Interface'''
 +
 +
{|{{BMTableStyle}}
 +
|-{{BMTHStyle}}
 +
! Test Description
 +
! Test Status
 +
  Windows XP SP2
 +
! Test Status
 +
  RHES 5.0
 +
! Remarks
 +
 +
|-
 +
|Open COSMOSUI with 8090 port (Make sure all COSMOS Components are up and running at this port)
 +
#http://localhost:8090/COSMOSUI/?page=cosmos
 +
Expected Result:
 +
#Should display all 6 DataManagers properly
 +
|Passed
 +
|Passed
 +
|Found All 6 DataManagers under COSMOS Web UI
 +
 +
|-
 +
|Open COSMOSUI, Verify Properties of all DataManagers
 +
#StatDataManager
 +
#LoggingDataManager
 +
#Sample Repository with Registration
 +
#Sample Repository with Query
 +
#Resource Repository
 +
#AperiMdr
 +
Expected Result:
 +
#All properties should display the correct EPR with port 9080
 +
|Passed
 +
|Passed
 +
|All EPR's under properties window displayed correctly
 +
 +
|-
 +
|Test StatDataManager, "Statistical Report" generation:
 +
#StatDataManager : server 33 tomcat1181829221515 CBE
 +
Expected Result:
 +
#Statistical Report should get generated properly
 +
|Passed
 +
|Passed
 +
|Statistical Report generated successfully
 +
 +
|-
 +
|Test "LoggingDataManager", Top 10 Log Report generation:
 +
#LoggingDataManager : access_log.cbe.xml
 +
Expected Result:
 +
#Top 10 Log report should get generated properly
 +
|Passed
 +
|Passed
 +
|Top 10 Log Report generated successfully
 +
 +
|-
 +
|Test "Sample Repository with Registration",self registration
 +
#Select "Sample Repository with Registration"
 +
#Click on "Register Configuration Items..."
 +
#Select "Sample Repository with Registration" under registration window
 +
Expected Result
 +
#Either it should not be allowed or it should display appropriate error message
 +
|Failed
 +
|Failed
 +
|Registration successful, but no result data
 +
 +
|-
 +
|Test "Sample Repository with Registration",Resource Repository registration
 +
#Select "Sample Repository with Registration"
 +
#Click on "Register Configuration Items..."
 +
#Select "Resource Repository" under registration window
 +
#Expected Result
 +
#Either registration should not be allowed or it should display appropriate error message
 +
|Failed
 +
|Failed
 +
|Registration successful, but no result data
 +
 +
|-
 +
|Test "Sample Repository with Registration",AperiMdr registration
 +
#Select "Sample Repository with Registration"
 +
#Click on "Register Configuration Items..."
 +
#Select "AperiMdr" under registration window
 +
Expected Result
 +
#Either registration should not be allowed or it should display appropriate error message
 +
|Failed
 +
|Failed
 +
|Error "org.apache.axis2.AxisFault: AperiMDR error: relationshipTemplate 'all-relationships' is missing a recordConstraint"
 +
 +
|-
 +
|Test Resource Repository, with valid query
 +
# select "Resource Repository" and click on "Submit Query"
 +
# Provide valid query and submit
 +
Expected Result:
 +
#Query should execute successfully
 +
|Passed
 +
|Passed
 +
|Queries executed successfully with proper results
 +
 +
|-
 +
|Test Resource Repository, with invalid query
 +
# select "Resource Repository" and click on "Submit Query"
 +
# Provide invalid query and submit
 +
Expected Result:
 +
#Query should fail with appropriate error message
 +
|Passed
 +
|Passed
 +
|Query execution failed with appropriate error message
 +
 +
|-
 +
|Test AperiMDR with proper db configuration and with valid query
 +
# select "AperiMdr" and create valid query using Query Builder
 +
Expected Result:
 +
# Query should execute successfully
 +
|Passed
 +
|Passed
 +
|Using Query builder able to execute and validate queries properly
 +
 +
|-
 +
|Test AperiMDR with proper db configuration and with invalid query
 +
# select "AperiMdr" and create invalid query using Query Builder
 +
Expected Result:
 +
# Query execution should fail and display appropriate error message
 +
|Passed
 +
|Passed
 +
|Query execution failed and displayed error message
 +
 +
|-
 +
|Test AperiMDR with invalid db configuration and with valid query
 +
# select "AperiMdr" and create valid query using Query Builder
 +
Expected Result:
 +
# Query execution should fail with db connection error
 +
|Passed
 +
|Passed
 +
|Query failed with db connection error
 +
 +
|-
 +
|Test "Sample Repository with Query", with valid query
 +
# select "Sample Repository with Query"
 +
# Create a valid query from query builder and submit it
 +
Expected Result:
 +
#Query execution should be successful
 +
|Passed
 +
|Passed
 +
|Query executed successfully
 +
 +
|-
 +
|Test "Sample Repository with Query", with invalid query
 +
# select "Sample Repository with Query"
 +
# Create an invalid query from query builder and submit it
 +
Expected Result:
 +
#Should display Empty Query response under GraphResponseViewer
 +
|Passed
 +
|Passed
 +
|Empty Query response displayed under GraphResponseViewer
 +
|}
 +
 +
==i10 ERs - JUnits/Manual Tests Inspection ==
 +
 +
http://wiki.eclipse.org/Cosmos1.0Features#Iteration_10_Enhancements, QA has verified only following ER's as most of the others are documentation related.
 +
 +
{|{{BMTableStyle}}
 +
|-{{BMTHStyle}}
 +
! Bugzilla ID
 +
! ID Summary
 +
! JUnits Tests
 +
! Manual Tests 
 +
! Remarks
 +
|-
 +
|214153
 +
|Data Visualization Customization - Customized CMDBf Query Builder
 +
|NA
 +
|Available
 +
|Found Manual tests provided as part of End2End testing at http://wiki.eclipse.org/COSMOS_DEMO_i10
 +
 +
|-
 +
|214143
 +
|Predefined CMDBf Queries
 +
|NA
 +
|Available
 +
|Manual Tests provided as part of End2End testing at http://wiki.eclipse.org/COSMOS_DEMO_i10
 +
 +
|-
 +
|220334
 +
|Additional Record Types for the Graph Reponse Viewer
 +
|NA
 +
|Available
 +
|Found TPTP Manual tests defined under /org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite
 +
 +
|-
 +
|221554
 +
|Provide a simple view to show the service meta data from an MDR.
 +
|NA
 +
|Available
 +
|No explicit tests provided, however this has been covered as part of End2End tests defined at http://wiki.eclipse.org/COSMOS_DEMO_i10
 +
 +
|-
 +
|217303
 +
|Support partial registering and deregister configuration Items
 +
|NA
 +
|Available
 +
|No explicit tests provided, however this has been covered as part of End2End tests defined at http://wiki.eclipse.org/COSMOS_DEMO_i10
 +
 +
|-
 +
|221901
 +
|Service side responses should be cached to reduce network calls
 +
|Available
 +
|NA
 +
|No Manual tests found, seems to have JUnits, but unable to find it's location
 +
 +
|-
 +
|205825
 +
|Update SML validator implementation based on changes to the SML latest draft
 +
|NA
 +
|NA
 +
|This ER related to documentation
 +
 +
|-
 +
|222982
 +
|Widget factory should generate unique ids when instantiating widgets
 +
|NA
 +
|Available
 +
|No explicit tests found, I guess some tests under COSMOSUITests suite would cover this as well.
 +
 +
|-
 +
|220949
 +
|MDRs need to accept vanilla web service requests
 +
|NA
 +
|Available
 +
|No explicit tests provided, however this has been covered with End2End tests available at http://wiki.eclipse.org/COSMOS_DEMO_i10
 +
 +
|-
 +
|220952
 +
|Broker to manage MDRs that do not support WSA and WSRF
 +
|NA
 +
|NA
 +
|No Manual/Junits tests found for testing
 +
 +
|-
 +
|220955
 +
|deregister data managers from broker
 +
|NA
 +
|Available
 +
|covered under http://wiki.eclipse.org/COSMOS_DEMO_i10
 +
 +
|-
 +
|221324
 +
|Additional Record Types for the Graph Reponse Viewer - Widget
 +
|Available
 +
|Available
 +
|Found TPTP Manual tests : /org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite
 +
 +
|-
 +
|221322
 +
|Additional Record Types for the Graph Reponse Viewer
 +
|Available
 +
|Available
 +
|Found TPTP Manual tests : /org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite
 +
 
|}
 
|}
  
 
==Basic Operational efficiency tests ==
 
==Basic Operational efficiency tests ==
 +
 +
'''Configuration Tests'''
  
 
{|{{BMTableStyle}}
 
{|{{BMTableStyle}}
 
|-{{BMTHStyle}}
 
|-{{BMTHStyle}}
−
! Test Description
+
! Test Description
−
! Output
+
! Test Status
−
! Status
+
! Remarks
−
! Remarks
+
 
|-
 
|-
 +
|Testing the scenario where COSMOSUI running on windows and all COSMOS services running on Linux
 +
#Install COSMOS service components under linux box
 +
#Install COSMOSUI,COSMOSReportViewer webapps application under windows
 +
#Modify BROKER EPR value under <COSMOSUI>/Configuration.properties file, pointing to linux hostname
 +
#From windows, open http://localhost:8080/COSMOSUI
 +
 +
Expected Result:
 +
#COSMOS UI should display all 6 Data Managers and all it's operations should work normally
 +
|Passed
 +
|COSMOSUI from windows is working fine.
 +
 +
|-
 +
|Testing the scenario where COSMOSUI running on linux and all COSMOS services running on windows
 +
#Install COSMOS service components under windows
 +
#Install COSMOSUI,COSMOSReportViewer webapps application under linux
 +
#Modify BROKER EPR value under <COSMOSUI>/Configuration.properties file, pointing to windows hostname
 +
#From linux, open http://localhost:8080/COSMOSUI
 +
 +
Expected Result:
 +
#COSMOS UI should display all 6 Data Managers and all it's operations should work normally
 +
|Passed
 +
|COSMOSUI from linux is working fine.
 
|}
 
|}
  
−
==i10 Test Execution Results ==
+
'''Concurrency Tests'''
 +
 
 +
{|{{BMTableStyle}}
 +
|-{{BMTHStyle}}
 +
! Test Description
 +
! Test Status
 +
! Remarks
 +
 
 +
|-
 +
|Concurrency tests, where multiple COSMOSUI's running on one machine
 +
#Install all COSMOS components on windows PC
 +
#open 2 COSMOSUI's under IE and 2 COSMOSUI's under Firebox browsers
 +
#Perform various test operations - Generating Stat/Log reports on all COSMOSUI's simultaneously,Executing CMDBf queries on all COSMOSUI's simultaneously
 +
 
 +
Expected Result:
 +
#All COSMOS UI's should work normally and should get appropriate results without any errors
 +
|Passed
 +
|All COSMOS UI's working normally
 +
 
 +
|-
 +
|Concurrency tests, where multiple COSMOSUI's and multiple COSMOS Command Line Clients running on one machine
 +
#Install all COSMOS components on windows PC
 +
#open 1 COSMOSUI under IE and 1 COSMOSUI under Firebox browser
 +
#open 2 COSMOS Command Line Shells
 +
#Perform various test operations - Execute CMDBf queries on all COSMOS Clients simultaneously
 +
 
 +
Expected Result:
 +
#All COSMOS clients should work normally and should get appropriate results without any errors
 +
|Passed
 +
|All COSMOS clients working properly
 +
|}
 +
 
 +
'''Volume/Performance Tests '''
 +
 
 +
 
 +
'''Apache JMeter Tool - Setup Instructions'''
 +
#Open Jmeter window (jmeter-install-dir\bin\jmeter.bat)
 +
#On the left side, right click on the ‘Test Plan’ node and select ‘Add->Thread Group’. This inserts one child node called ‘Thread Group’ under ‘Test Plan’.
 +
#Select the ‘Thread Group’ node. On the right side for ‘Number of Threads’ text box, give the number of samples.
 +
#Right click on ‘Thread Group’ node, and select ‘Add->Sampler->SOAP/XML- RPC Request’. This inserts one child node called ‘SOAP/XML-RPC Request’ under ‘Thread Group’
 +
#Select the ‘SOAP/XML-RPC Request’ node. On the right-side, in text box URL: give the URL to which the request to be send. In the ‘Soap/XML-RPC Data’ text area type the ‘SOAP request’
 +
#Right click on ‘Thread Group’ node, and select ‘Add->Listener->Summary Report’. This inserts one child node called ‘Summary Report’ under the ‘Thread Group’. We can select our own format for displaying results (summary, table, tree…etc)
 +
#Save the test
 +
#Now, to run the test, from menu bar select Run->Start for running a test.
 +
#Run->Stop for stopping  test
 +
#The results will be populated in the Listeners that we have added to the test.
 +
 
 +
 
 +
{|{{BMTableStyle}}
 +
|-{{BMTHStyle}}
 +
! Test Description
 +
! Test Status
 +
! Remarks
 +
 
 +
|-
 +
|Using Apache JMeter Tool,testing Broker with samples of 10,50,100,150,200 users simulation
 +
#Follow the procedure to create the test, running the test using JMeter.
 +
#Provide below SOAP request of Broker to Jmeter tool and run it with users simulation of 10,50,100,150,200.
 +
* URL = http://localhost:8080/axis2/services/Broker
 +
* Soap/XML-RPC Data = <?xml version='1.0' encoding='utf-8'?>
 +
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 +
<soapenv:Body><tns:getAllGroups xmlns:tns="http://www.eclipse.org/cosmos/broker" />
 +
</soapenv:Body></soapenv:Envelope>
 +
 
 +
Expected Result:
 +
#Broker requests with all above user samples should work fine with appropriate turn around time
 +
|Passed
 +
|None
 +
|-
 +
|Using Apache JMeter Tool,testing StatDataManager with samples of 10,50,100,150,200 users simulation
 +
#Follow the procedure to create the test, running the test using JMeter.
 +
#Provide below SOAP request of StatDataManager to Jmeter tool and run it with users simulation of 10,50,100,150,200.
 +
* URL = http://localhost:8080/axis2/services/StatisticalDataManager
 +
* Soap/XML-RPC Data = <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><tns:getDatasourceKeysets xmlns:tns="http://axis.datamgr.stat.dc.cosmos.eclipse.org" /></soapenv:Body></soapenv:Envelope>
 +
 
 +
Expected Result:
 +
#StatDataManager requests with all above user samples should work fine with appropriate turn around time
 +
|Passed
 +
|None
 +
 
 +
|-
 +
|Using Apache JMeter Tool,testing LoggingDataManager with samples of 10,50,100,150,200 users simulation
 +
#Follow the procedure to create the test, running the test using JMeter.
 +
#Provide below SOAP request of LoggingDataManager to Jmeter tool and run it with users simulation of 10,50,100,150,200.
 +
* URL = http://localhost:8080/axis2/services/CBEServices
 +
* Soap/XML-RPC Data = <?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><tns:getLogFileIdentifiers xmlns:tns="http://axis.datamgr.logging.dc.cosmos.eclipse.org" /></soapenv:Body></soapenv:Envelope>
 +
 
 +
Expected Result:
 +
#LoggingDataManager requests with all above user samples should work fine with appropriate turn around time
 +
|Passed
 +
|None
 +
 
 +
|-
 +
|Using Apache JMeter Tool,testing FederatingCMDB with samples of 10,50,100,150,200 users simulation
 +
#Follow the procedure to create the test, running the test using JMeter.
 +
#Provide below SOAP requests of FederatingCMDB to Jmeter tool and run it with users simulation of 10,50,100,150,200.
 +
 
 +
*URL=http://localhost:8080/axis2/services/Broker
 +
*Soap/XML-RPC Data=<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><tns:getGroupsByNamespace xmlns:tns="http://www.eclipse.org/cosmos/broker"><tns:namespace>http://cmdbf.org/schema/1-0-0/query</tns:namespace></tns:getGroupsByNamespace></soapenv:Body></soapenv:Envelope>
 +
 
 +
 
 +
*URL=http://localhost:8080/axis2/services/ExampleMdrQueryService
 +
*Soap/XML-RPC Data=<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope"><soapenv:Body><cmdbf:query xmlns:cmdbf="http://cmdbf.org/schema/1-0-0/datamodel">
 +
<cmdbf:itemTemplate id="all-items">
 +
</cmdbf:itemTemplate>
 +
<cmdbf:relationshipTemplate id="all-relationships">
 +
<cmdbf:sourceTemplate ref="all-items" />
 +
<cmdbf:targetTemplate ref="all-items" />
 +
</cmdbf:relationshipTemplate>
 +
</cmdbf:query></soapenv:Body></soapenv:Envelope>
 +
 
 +
 
 +
*URL=http://localhost:8080/axis2/services/TestFederatingCMDBRegistrationService
 +
*Soap Data=<?xml version='1.0' encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Body><cmdbf:registerRequest xmlns:cmdbf="http://cmdbf.org/schema/1-0-0/datamodel">
 +
<cmdbf:mdrId />
 +
<cmdbf:itemList>
 +
<cmdbf:item>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<student>
 +
 +
<identity firstName="Mike" id="03" lastName="Lee">
 +
</identity>
 +
 +
</student>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>03</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>03</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:item>
 +
<cmdbf:item>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<student>
 +
 +
<identity firstName="Jane" id="02" lastName="Ryerson">
 +
</identity>
 +
 +
</student>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>02</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>02</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:item>
 +
<cmdbf:item>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<student>
 +
 +
<identity firstName="Bob" id="01" lastName="Davidson">
 +
</identity>
 +
 +
</student>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>01</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>01</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:item>
 +
<cmdbf:item>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<teacher>
 +
 +
<identity firstName="Heather" id="staff02" lastName="Reeba">
 +
</identity>
 +
 +
</teacher>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>staff02</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>staff02</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:item>
 +
<cmdbf:item>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<teacher>
 +
 +
<identity firstName="Dawn" id="staff01" lastName="Johnson">
 +
</identity>
 +
 +
</teacher>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>staff01</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>staff01</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:item>
 +
</cmdbf:itemList>
 +
<cmdbf:relationshipList>
 +
<cmdbf:relationship>
 +
<cmdbf:source>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>staff02</cmdbf:localId>
 +
</cmdbf:source>
 +
<cmdbf:target>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>03</cmdbf:localId>
 +
</cmdbf:target>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<class courseCode="PHY01" name="Physics">
 +
 +
<students>
 +
 +
<enrolledStudent idRef="02">
 +
</enrolledStudent>
 +
 +
<enrolledStudent idRef="03">
 +
</enrolledStudent>
 +
 +
</students>
 +
 +
<teacher idRef="staff02">
 +
</teacher>
 +
 +
</class>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>PHY01</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>PHY01</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:relationship>
 +
<cmdbf:relationship>
 +
<cmdbf:source>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>staff02</cmdbf:localId>
 +
</cmdbf:source>
 +
<cmdbf:target>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>02</cmdbf:localId>
 +
</cmdbf:target>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<class courseCode="MAT01" name="Mathematics">
 +
 +
<students>
 +
 +
<enrolledStudent idRef="01">
 +
</enrolledStudent>
 +
 +
<enrolledStudent idRef="02">
 +
</enrolledStudent>
 +
 +
</students>
 +
 +
<teacher idRef="staff02">
 +
</teacher>
 +
 +
</class>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>MAT01</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>MAT01</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:relationship>
 +
<cmdbf:relationship>
 +
<cmdbf:source>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>staff02</cmdbf:localId>
 +
</cmdbf:source>
 +
<cmdbf:target>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>02</cmdbf:localId>
 +
</cmdbf:target>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<class courseCode="PHY01" name="Physics">
 +
 +
<students>
 +
 +
<enrolledStudent idRef="02">
 +
</enrolledStudent>
 +
 +
<enrolledStudent idRef="03">
 +
</enrolledStudent>
 +
 +
</students>
 +
 +
<teacher idRef="staff02">
 +
</teacher>
 +
 +
</class>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>PHY01</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>PHY01</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:relationship>
 +
<cmdbf:relationship>
 +
<cmdbf:source>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>staff02</cmdbf:localId>
 +
</cmdbf:source>
 +
<cmdbf:target>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>01</cmdbf:localId>
 +
</cmdbf:target>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<class courseCode="MAT01" name="Mathematics">
 +
 +
<students>
 +
 +
<enrolledStudent idRef="01">
 +
</enrolledStudent>
 +
 +
<enrolledStudent idRef="02">
 +
</enrolledStudent>
 +
 +
</students>
 +
 +
<teacher idRef="staff02">
 +
</teacher>
 +
 +
</class>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>MAT01</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>MAT01</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:relationship>
 +
<cmdbf:relationship>
 +
<cmdbf:source>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>staff01</cmdbf:localId>
 +
</cmdbf:source>
 +
<cmdbf:target>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>03</cmdbf:localId>
 +
</cmdbf:target>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<class courseCode="ECM01" name="Economics">
 +
 +
<students>
 +
 +
<enrolledStudent idRef="01">
 +
</enrolledStudent>
 +
 +
<enrolledStudent idRef="03">
 +
</enrolledStudent>
 +
 +
</students>
 +
 +
<teacher idRef="staff01">
 +
</teacher>
 +
 +
</class>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>ECM01</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>ECM01</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:relationship>
 +
<cmdbf:relationship>
 +
<cmdbf:source>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>staff01</cmdbf:localId>
 +
</cmdbf:source>
 +
<cmdbf:target>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>01</cmdbf:localId>
 +
</cmdbf:target>
 +
<cmdbf:record xmlns="http://school">
 +
 +
<class courseCode="ECM01" name="Economics">
 +
 +
<students>
 +
 +
<enrolledStudent idRef="01">
 +
</enrolledStudent>
 +
 +
<enrolledStudent idRef="03">
 +
</enrolledStudent>
 +
 +
</students>
 +
 +
<teacher idRef="staff01">
 +
</teacher>
 +
 +
</class>
 +
<cmdbf:recordMetadata>
 +
<cmdbf:recordId>ECM01</cmdbf:recordId>
 +
</cmdbf:recordMetadata>
 +
</cmdbf:record>
 +
<cmdbf:instanceId>
 +
<cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId>
 +
<cmdbf:localId>ECM01</cmdbf:localId>
 +
</cmdbf:instanceId>
 +
</cmdbf:relationship>
 +
</cmdbf:relationshipList>
 +
</cmdbf:registerRequest></soapenv:Body></soapenv:Envelope>
 +
 
 +
Expected Result:
 +
#FederatingCMDB requests with all above user samples should work fine with appropriate turn around time
 +
|Passed
 +
|None
 +
 
 +
|-
 +
|Using Apache JMeter Tool,testing ExampleMDR with samples of 10,50,100,150,200 users simulation
 +
#Follow the procedure to create the test, running the test using JMeter.
 +
#Provide below SOAP request of ExampleMDR to Jmeter tool and run it with users simulation of 10,50,100,150,200.
 +
*URL=http://localhost:8080/axis2/services/ExampleMdrQueryService
 +
*Soap Data=<?xml version='1.0' encoding='utf-8'?>
 +
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 +
<soapenv:Body>
 +
<st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel">
 +
<st:itemTemplate id="items" suppressFromResult="false">
 +
<st:recordConstraint>
 +
<st:recordType localName="student" namespace="http://schemas.examplemdr.org/schema/ExampleMDR" />
 +
</st:recordConstraint></st:itemTemplate></st:query>
 +
</soapenv:Body>
 +
</soapenv:Envelope>
 +
 
 +
Expected Result:
 +
#ExampleMDR requests with all above user samples should work fine with appropriate turn around time
 +
|Passed
 +
|None
 +
 
 +
|-
 +
|Using Apache JMeter Tool,testing SMLMDR with samples of 10,50,100,150,200 users simulation
 +
#Follow the procedure to create the test, running the test using JMeter.
 +
#Provide below SOAP request of SMLMDR to Jmeter tool and run it with user samples of 10,50,100,150,200
 +
*URL=http://localhost:8080/axis2/services/SmlMdrQueryService
 +
*SOAP Data=<?xml version='1.0' encoding='utf-8'?>
 +
<soapenv:Envelope
 +
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 +
<soapenv:Body>
 +
<st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel">
 +
<st:itemTemplate id="cableTemplate">
 +
<st:recordConstraint>
 +
<st:propertyValue localName="format" namespace="propertyModel">
 +
<st:equal>RJ45</st:equal>
 +
</st:propertyValue>
 +
</st:recordConstraint>
 +
</st:itemTemplate>
 +
</st:query>
 +
</soapenv:Body>
 +
</soapenv:Envelope>
 +
 
 +
Expected Result:
 +
#SMLMDR requests with all above user samples should work fine with appropriate turn around time
 +
|Passed
 +
|None
 +
 
 +
|-
 +
|Using Apache JMeter Tool,testing AperiMdr with samples of 10,50,100,150,200 users simulation
 +
#Follow the procedure to create the test, running the test using JMeter.
 +
#Provide below SOAP request of AperiMdr to Jmeter tool and run it with 10,50,100,150,200 users simulation
 +
*URL=http://localhost:8080/axis2/services/AperiMdrQueryService
 +
*Soap Data=<?xml version='1.0' encoding='utf-8'?>
 +
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 +
<soapenv:Body>
 +
<st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel">
 +
<st:itemTemplate id="items" suppressFromResult="false">
 +
<st:recordConstraint>
 +
<st:recordType localName="StoragePool" namespace="http://schemas.aperimdr.org/schema/AperiMDR" />
 +
</st:recordConstraint>
 +
</st:itemTemplate>
 +
</st:query>
 +
</soapenv:Body>
 +
</soapenv:Envelope>
 +
 
 +
Expected Result:
 +
#AperiMdr requests with all above user samples should work fine with appropriate turn around time
 +
|Passed
 +
|None
 +
|}
 +
 
 +
 
 +
'''Scalability/Stability Tests'''
 +
 
 +
{|{{BMTableStyle}}
 +
|-{{BMTHStyle}}
 +
! Test Description
 +
! Test Status
 +
! Remarks
 +
 
 +
|-
 +
|Using Apache JMeter Tool,testing ExampleMDR with multiple volumes of test data (data.xml) 1MB,
 +
2MB,2.5MB with user samples of 10,50,100,150,200.
 +
#Modify data.xml file to insert test data with 5000,10000,15000 student records
 +
#Follow the procedure to create the test, running the test using JMeter.
 +
#Provide below SOAP request of ExampleMDR to Jmeter tool and run it with 10,50,100,150,200 users simulation
 +
*URL=http://localhost:8080/axis2/services/ExampleMdrQueryService
 +
*Soap Data=<?xml version='1.0' encoding='utf-8'?>
 +
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 +
<soapenv:Body>
 +
<st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel">
 +
<st:itemTemplate id="items" suppressFromResult="false">
 +
<st:recordConstraint>
 +
<st:recordType localName="student" namespace="http://schemas.examplemdr.org/schema/ExampleMDR" />
 +
</st:recordConstraint></st:itemTemplate></st:query>
 +
</soapenv:Body>
 +
</soapenv:Envelope>
 +
 
 +
Expected Result:
 +
#ExampleMDR requests with all above user samples should work fine with appropriate turn around time
 +
|Passed
 +
|Found "Java Heap Space" and "Post too Large" configuration issues while testing with 2MB data.mxl file
 +
|}
 +
 
 +
==QA Test Results ==
 +
 
 +
* DataVisualization.UI.Component.Widget_1209468870674.execution 
 +
http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/common/org.eclipse.cosmos.examples.e2e.tests/manual-results/DataVisualization.UI.Component.Widget_1209468870674.execution?root=Technology_Project&view=log
 +
* DataVisualization.UI.Reports_1209465105913.execution  http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/common/org.eclipse.cosmos.examples.e2e.tests/manual-results/DataVisualization.UI.Reports_1209465105913.execution?root=Technology_Project&view=log
 +
* Performance & Scalability Tests
 +
http://wiki.eclipse.org/images/9/90/Cosmos_i10_performancetestResults.zip

Latest revision as of 07:03, 7 May 2008

This page is created by QA to track all i10 test execution activities from 21 Apr'08 to 02 May'08. This page will be filled in by the QA team as i10 testing phase proceeds.

QA Test Execution Activities for i10

Primary Tasks:

  1. E2E testing on Windows XP, 2003 and Red Hat Enterprise Linux Server 5.0 (http://wiki.eclipse.org/COSMOS_DEMO_i10 )
  2. i10 ERs – JUnits/Manual Tests inspection (http://wiki.eclipse.org/Cosmos1.0Features#Iteration_10_Enhancements)
  3. Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite / UI.Reports.testsuite)

Secondary Tasks:

  1. Execution of i10 supplement e2e tests (negative) from QA.
  2. Basic operational efficiency tests (limited performance tests)

For complete i10 QA Activities, refer http://wiki.eclipse.org/COSMOS_QA_i10_Activities

QA Status Summary

Iteration 10 QA testing effort and completion status (21 Apr’08 – 02 May’08)

  • QA announces the finish of our testing effort for COSMOS iteration 10 with 1 day ahead of scheduled completion date (02 May, 2008)
  • The following activities were completed by QA, as per defined scope of i10 QA activities (http://wiki.eclipse.org/COSMOS_QA_i10_Activities)

i10 QA Execution Summary:

  1. End2End testing on 3 platforms (Windows XP,2003 and Red Hat Enterprise Server 5.0) based on i10 demo page instructions (http://wiki.eclipse.org/COSMOS_DEMO_i10)
  2. Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite and DataVisualization.UI.Reports.testsuite)
  3. Review of i10 ER’s for JUnits and Manual Tests
  4. Additional E2E tests execution (Negative tests) (http://wiki.eclipse.org/COSMOS_i10_QA_Phase_Execution#i10_End2End_Supplement_Tests_.28Negative.29)
  5. Limited Performance Tests (Configuration tests & Concurrency tests)

i10 QA Execution Status Page:

i10 bugs list filed by QA:

E2E testing status (WinXP SP2/Win2K3 SP1/RHES 5.0)

Build Installation and Configuration

Test Description Test Status 
 Windows XP SP2
Test Status 
 Windows 2003 SP1
Test Status 
 RHES 5.0
Remarks
<cosmos-demo>/bin/configDemo.bat <tomcatDir> <cosmosDir> <birt-path> <dojo-path> <axis2war-path> Passed Passed Passed None

Starting COSMOS components : Management Console

Test Description Test Status 
 Windows XP SP2
Test Status 
 Windows 2003 SP1
Test Status 
 RHES 5.0
Remarks
<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:8080/axis2/services/Broker Passed Passed Passed None
COSMOS> broker registerAllDataManagers Passed Passed Passed Found warnings - "log4j:WARN No appenders could be found for logger (org.apache.axis2.util.Loader).

log4j:WARN Please initialize the log4j system properly."

COSMOS> broker getDataManagers Passed Passed Passed None
COSMOS> finddatamanagers http://localhost:8080/axis2/services/CosmosServiceFinder Passed Passed Passed None
COSMOS> broker deregister domsr01-xp StatDataManager Passed Passed Passed None
COSMOS> broker deregister domsr01-xp LoggingDataManager Passed Passed Passed None
COSMOS> broker deregister domsr01-xp TestFederatingCMDB Passed Passed Passed None
COSMOS> broker deregister domsr01-xp ExampleMdr Passed Passed Passed None
COSMOS> broker deregister domsr01-xp SMLMDR Passed Passed Passed None
COSMOS> broker deregister domsr01-xp AperiMdr Passed Passed Passed None
COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testPropertyQuery.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-students.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-teachers.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\relationship-all-classes.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\teaches-relationship.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_localid.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_display_name.txt Passed Passed Passed None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_relationship.txt Passed Passed Passed None
COSMOS> COSMOS> graphQuery domsr01-xp TestFederatingCMDB D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\TestFederatingCMDB\all_items.txt Passed Passed Passed Empty Response, no data found.

Using the COSMOS Web User Interface

Test Description Test Status 
 Windows XP SP2
Test Status 
 Windows 2003 SP1
Test Status 
 RHES 5.0
Remarks
http://localhost:8080/COSMOSUI/?page=cosmos Passed Passed Passed Found All 6 DataManagers under COSMOS Web UI
StatDataManager

server 33 tomcat1181829221515 CBE

Passed Passed Passed Statistical Report generated successfully
StatDataManager

server 33 tomcat1181829221515 TomcatSet

Passed Passed Passed Statistical Report generated successfully
LoggingDataManager

access_log.cbe.xml

Passed Passed Passed Top 10 Log Report generated successfully
LoggingDataManager

windows_app_log.cbe.xml

Passed Passed Passed Top 10 Log Report generated successfully
Sample Repository with Registration

Submit Query

Passed Passed Passed Query for all-items resulted Empty Reponse, no data found.
Sample Repository with Registration

Register Configuration Items...

Passed Passed Passed Registration with Example MDR is successful
Sample Repository with Registration

Deregister Configuration Items...

Passed Passed Passed Deregistration with Example MDR is successful
Sample Repository with Registration

Submit Query

Passed Passed Passed Query for all-items resulted Empty Reponse, no data found.
Sample Repository with Registration

View MetaData

Passed Passed Passed No Metadata available
Resource Repository

Submit Query

Passed Passed Passed Queries executed successfully with proper results
Resource Repository

View MetaData

Passed Passed Passed No Metadata available
AperiMDR

Submit Query

Passed Passed Passed Using Query builder able to execute and validate queries properly
AperiMDR

View MetaData

Passed Passed Passed Displayed MetaData properly
Sample Repository with Query

Get All Students

Passed Passed Passed Displayed Query Result for Students data correctly
Sample Repository with Query

Get All Teachers

Passed Passed Passed Displayed Query Result for Teachers data correctly
Sample Repository with Query

Get All Classes

Passed Passed Passed Displayed Query Result for Classes data correctly
Sample Repository with Query

Staff01

Passed Passed Passed Displayed data for 'students taught by teacher-staff01' correctly
Sample Repository with Query

Staff02

Passed Passed Passed Displayed data for 'students taught by teacher-staff02' correctly
Sample Repository with Query

Submit Query

Passed Passed Passed Using Query Builder, able to create and execute queries correctly
Sample Repository with Query

View MetaData

Passed Passed Passed Displayed MetaData information correctly

Data Visualization TPTP Manual Tests

/org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite

Test Description Test Status 
 Windows XP SP2
Test Status 
 RHES 5.0
Remarks
navigate Passed Passed 6 nodes should be shown under "Data Managers"

No "Monitoring Data Repository" node, it should be "StatDataManager" node

navigate2 Passed Passed No "Create Query" found, it should be "Submit Query"
properties Passed Passed Given EPR is wrong, it should be http://localhost:8080/axis2/services/ExampleMdrDataManager
visualize log report Passed Passed No "Monitoring Data Repository" available, it should be "StatDataManager
visualize stat report Passed Passed None
properties 2 Passed Passed None
example cmdbf query Passed Passed XML output contents seems to be wrong, actual result shows only one student detail - "firstName="Mike" id="03 lastname="Lee""
SML CMDBF Graph Response Viewer Passed Passed None
Example MDR CMDBF Graph Response Viewer Passed Passed No cancel button available under details view window, it should be OK button.
Multiple Graph Response Viewers Passed Passed No "Cancel" button available under detail area, it should be Ok button
Registration Example Passed Passed None
DeRegistration Example Passed Passed None
Partial Registration of MDR with CMDBf Passed Passed None
Partial DeRegistration of MDR with CMDBf Passed Passed None

/org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Reports.testsuite

Test Description Test Status 
 Windows XP SP2
Test Status 
 RHES 5.0
Remarks
LogReport Passed Passed None
StatReport Passed Passed None

i10 End2End Supplement Tests (Negative)

COSMOS Command Line Client : Management Console

Test Description Test Status 
 Windows XP SP2
Test Status 
 RHES 5.0
Remarks
Configure tomcat port to 9080 (other than default port : 8080) and test the following:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker

Expected Result:

  1. Should display COSMOS prompt without any errors
 Passed Passed None
Test Broker registration other than defaul port 8080 (say 9080):
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder

Expected Result:

  1. All DataManagers should successfuly get registered to broker on port 9080
 Failed Failed DataManagers are getting registered on default port 8080 only, ignored actual port 9080
Test Broker registration multiple times:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder
  3. COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder

Expected Result:

  1. Under tomcat console window, it should display "Service has already been registered at this broker. No new brokers are register

ed."

Passed Passed Found expected message under tomcat console window
Without broker registration, test the following:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker getDataManagers

Expected Result:

  1. Should display empty list of DataManagers
 Passed Passed None
After succesful broker registration, test the following:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker getDataManagers

Expected Result:

  1. Should display all 6 DataManagers registered with broker
 Passed Passed None
Without any broker registration, test the following
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> finddatamanagers http://localhost:9080/axis2/services/CosmosServiceFinder

Expected Result:

  1. Should display all 6 DataManagers deployed under tomcat with port 9080
 Passed Passed None
Test deregistration of StatDataManager with valid hostname and valid datamanager name :
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker deregister domsr01-xp StatDataManager

Expected Result:

  1. Deregistration should be successful
 Passed Passed None
Test deregistration of StatDataManager with invalid hostname and valid datamanager name :
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker deregister unknown StatDataManager

Expected Result:

  1. Deregistration should fail and should display appropriate error message
 Failed Failed No Failure message, it still shows "DataManager deregistered"
Test deregistration of StatDataManager with valid hostname and invalid datamanager name :
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> broker deregister domsr01-xp StatDataMgr

Expected Result:

  1. Deregistration should fail and should display appropriate error message
 Failed Failed No Failure message, it still shows "DataManager deregistered"
Test graphQuery for SMLMDR, with valid hostname,valid DataManager,valid query path:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt

Expected Result:

  1. Query execution should be succesful.
 Passed Passed This is depended on Broker registration issue, fixed manually to test this.
Test graphQuery for SMLMDR, with invalid hostname,valid DataManager,valid query path:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery unknown SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt

Expected Result:

  1. Query execution should fail and throw appropriate error message.
 Passed Passed Displaying error message as "SMLMDR is not registered at the broker."
Test graphQuery for SMLMDR, with valid hostname,invalid DataManager,valid query path:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp StatDataManager D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt

Expected Result:

  1. Query execution should fail and throw appropriate error message.
 Passed Passed Displayed error message as "StatDataManager is not registered at the broker."
Test graphQuery for SMLMDR, with valid hostname,valid DataManager,invalid query path:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDRTest\testInstanceIdQuery.txt

Expected Result:

  1. Query execution should fail and throw appropriate error message.
 Passed Passed Displayed "java.io.FileNotFoundException:" error message tree
Test graphQuery for SMLMDR, with valid hostname,valid DataManager,valid query path with spaces:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp SMLMDR "D:\cosmos i10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt"

Expected Result:

  1. Query execution should be successful
 Failed Failed Displayed "java.io.FileNotFoundException:" error message tree
Test graphQuery for AperMdr with valid data, with valid Aperi database settings:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt

Expected Result:

  1. Query should execute successfully
 Passed Passed Query executed successfully
Test graphQuery for AperMdr with invalid query data, with valid Aperi database settings:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt

Expected Result:

  1. Query execution should fail and display appropriate error message
 Passed Passed Query execution failed with appropriate error message
Test graphQuery for AperMdr with valid data, with invalid Aperi database settings:
  1. <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker
  2. COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt

Expected Result:

  1. Query execution should fail, with DB Connection error
 Passed Passed Displayed "aperiDB.getStorageSystems error"

Using the COSMOS Web User Interface

Test Description Test Status 
 Windows XP SP2
Test Status 
 RHES 5.0
Remarks
Open COSMOSUI with 8090 port (Make sure all COSMOS Components are up and running at this port)
  1. http://localhost:8090/COSMOSUI/?page=cosmos

Expected Result:

  1. Should display all 6 DataManagers properly
 Passed Passed Found All 6 DataManagers under COSMOS Web UI
Open COSMOSUI, Verify Properties of all DataManagers
  1. StatDataManager
  2. LoggingDataManager
  3. Sample Repository with Registration
  4. Sample Repository with Query
  5. Resource Repository
  6. AperiMdr

Expected Result:

  1. All properties should display the correct EPR with port 9080
 Passed Passed All EPR's under properties window displayed correctly
Test StatDataManager, "Statistical Report" generation:
  1. StatDataManager : server 33 tomcat1181829221515 CBE

Expected Result:

  1. Statistical Report should get generated properly
 Passed Passed Statistical Report generated successfully
Test "LoggingDataManager", Top 10 Log Report generation:
  1. LoggingDataManager : access_log.cbe.xml

Expected Result:

  1. Top 10 Log report should get generated properly
 Passed Passed Top 10 Log Report generated successfully
Test "Sample Repository with Registration",self registration
  1. Select "Sample Repository with Registration"
  2. Click on "Register Configuration Items..."
  3. Select "Sample Repository with Registration" under registration window

Expected Result

  1. Either it should not be allowed or it should display appropriate error message
 Failed Failed Registration successful, but no result data
Test "Sample Repository with Registration",Resource Repository registration
  1. Select "Sample Repository with Registration"
  2. Click on "Register Configuration Items..."
  3. Select "Resource Repository" under registration window
  4. Expected Result
  5. Either registration should not be allowed or it should display appropriate error message
 Failed Failed Registration successful, but no result data
Test "Sample Repository with Registration",AperiMdr registration
  1. Select "Sample Repository with Registration"
  2. Click on "Register Configuration Items..."
  3. Select "AperiMdr" under registration window

Expected Result

  1. Either registration should not be allowed or it should display appropriate error message
 Failed Failed Error "org.apache.axis2.AxisFault: AperiMDR error: relationshipTemplate 'all-relationships' is missing a recordConstraint"
Test Resource Repository, with valid query
  1. select "Resource Repository" and click on "Submit Query"
  2. Provide valid query and submit

Expected Result:

  1. Query should execute successfully
 Passed Passed Queries executed successfully with proper results
Test Resource Repository, with invalid query
  1. select "Resource Repository" and click on "Submit Query"
  2. Provide invalid query and submit

Expected Result:

  1. Query should fail with appropriate error message
 Passed Passed Query execution failed with appropriate error message
Test AperiMDR with proper db configuration and with valid query
  1. select "AperiMdr" and create valid query using Query Builder

Expected Result:

  1. Query should execute successfully
 Passed Passed Using Query builder able to execute and validate queries properly
Test AperiMDR with proper db configuration and with invalid query
  1. select "AperiMdr" and create invalid query using Query Builder

Expected Result:

  1. Query execution should fail and display appropriate error message
 Passed Passed Query execution failed and displayed error message
Test AperiMDR with invalid db configuration and with valid query
  1. select "AperiMdr" and create valid query using Query Builder

Expected Result:

  1. Query execution should fail with db connection error
 Passed Passed Query failed with db connection error
Test "Sample Repository with Query", with valid query
  1. select "Sample Repository with Query"
  2. Create a valid query from query builder and submit it

Expected Result:

  1. Query execution should be successful
 Passed Passed Query executed successfully
Test "Sample Repository with Query", with invalid query
  1. select "Sample Repository with Query"
  2. Create an invalid query from query builder and submit it

Expected Result:

  1. Should display Empty Query response under GraphResponseViewer
 Passed Passed Empty Query response displayed under GraphResponseViewer

i10 ERs - JUnits/Manual Tests Inspection

http://wiki.eclipse.org/Cosmos1.0Features#Iteration_10_Enhancements, QA has verified only following ER's as most of the others are documentation related.

Bugzilla ID ID Summary JUnits Tests Manual Tests Remarks
214153 Data Visualization Customization - Customized CMDBf Query Builder NA Available Found Manual tests provided as part of End2End testing at http://wiki.eclipse.org/COSMOS_DEMO_i10
214143 Predefined CMDBf Queries NA Available Manual Tests provided as part of End2End testing at http://wiki.eclipse.org/COSMOS_DEMO_i10
220334 Additional Record Types for the Graph Reponse Viewer NA Available Found TPTP Manual tests defined under /org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite
221554 Provide a simple view to show the service meta data from an MDR. NA Available No explicit tests provided, however this has been covered as part of End2End tests defined at http://wiki.eclipse.org/COSMOS_DEMO_i10
217303 Support partial registering and deregister configuration Items NA Available No explicit tests provided, however this has been covered as part of End2End tests defined at http://wiki.eclipse.org/COSMOS_DEMO_i10
221901 Service side responses should be cached to reduce network calls Available NA No Manual tests found, seems to have JUnits, but unable to find it's location
205825 Update SML validator implementation based on changes to the SML latest draft NA NA This ER related to documentation
222982 Widget factory should generate unique ids when instantiating widgets NA Available No explicit tests found, I guess some tests under COSMOSUITests suite would cover this as well.
220949 MDRs need to accept vanilla web service requests NA Available No explicit tests provided, however this has been covered with End2End tests available at http://wiki.eclipse.org/COSMOS_DEMO_i10
220952 Broker to manage MDRs that do not support WSA and WSRF NA NA No Manual/Junits tests found for testing
220955 deregister data managers from broker NA Available covered under http://wiki.eclipse.org/COSMOS_DEMO_i10
221324 Additional Record Types for the Graph Reponse Viewer - Widget Available Available Found TPTP Manual tests : /org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite
221322 Additional Record Types for the Graph Reponse Viewer Available Available Found TPTP Manual tests : /org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite

Basic Operational efficiency tests

Configuration Tests

Test Description Test Status Remarks
Testing the scenario where COSMOSUI running on windows and all COSMOS services running on Linux
  1. Install COSMOS service components under linux box
  2. Install COSMOSUI,COSMOSReportViewer webapps application under windows
  3. Modify BROKER EPR value under <COSMOSUI>/Configuration.properties file, pointing to linux hostname
  4. From windows, open http://localhost:8080/COSMOSUI

Expected Result:

  1. COSMOS UI should display all 6 Data Managers and all it's operations should work normally
 Passed COSMOSUI from windows is working fine.
Testing the scenario where COSMOSUI running on linux and all COSMOS services running on windows
  1. Install COSMOS service components under windows
  2. Install COSMOSUI,COSMOSReportViewer webapps application under linux
  3. Modify BROKER EPR value under <COSMOSUI>/Configuration.properties file, pointing to windows hostname
  4. From linux, open http://localhost:8080/COSMOSUI

Expected Result:

  1. COSMOS UI should display all 6 Data Managers and all it's operations should work normally
 Passed COSMOSUI from linux is working fine.

Concurrency Tests

Test Description Test Status Remarks
Concurrency tests, where multiple COSMOSUI's running on one machine
  1. Install all COSMOS components on windows PC
  2. open 2 COSMOSUI's under IE and 2 COSMOSUI's under Firebox browsers
  3. Perform various test operations - Generating Stat/Log reports on all COSMOSUI's simultaneously,Executing CMDBf queries on all COSMOSUI's simultaneously

Expected Result:

  1. All COSMOS UI's should work normally and should get appropriate results without any errors
 Passed All COSMOS UI's working normally
Concurrency tests, where multiple COSMOSUI's and multiple COSMOS Command Line Clients running on one machine
  1. Install all COSMOS components on windows PC
  2. open 1 COSMOSUI under IE and 1 COSMOSUI under Firebox browser
  3. open 2 COSMOS Command Line Shells
  4. Perform various test operations - Execute CMDBf queries on all COSMOS Clients simultaneously

Expected Result:

  1. All COSMOS clients should work normally and should get appropriate results without any errors
 Passed All COSMOS clients working properly

Volume/Performance Tests


Apache JMeter Tool - Setup Instructions

  1. Open Jmeter window (jmeter-install-dir\bin\jmeter.bat)
  2. On the left side, right click on the ‘Test Plan’ node and select ‘Add->Thread Group’. This inserts one child node called ‘Thread Group’ under ‘Test Plan’.
  3. Select the ‘Thread Group’ node. On the right side for ‘Number of Threads’ text box, give the number of samples.
  4. Right click on ‘Thread Group’ node, and select ‘Add->Sampler->SOAP/XML- RPC Request’. This inserts one child node called ‘SOAP/XML-RPC Request’ under ‘Thread Group’
  5. Select the ‘SOAP/XML-RPC Request’ node. On the right-side, in text box URL: give the URL to which the request to be send. In the ‘Soap/XML-RPC Data’ text area type the ‘SOAP request’
  6. Right click on ‘Thread Group’ node, and select ‘Add->Listener->Summary Report’. This inserts one child node called ‘Summary Report’ under the ‘Thread Group’. We can select our own format for displaying results (summary, table, tree…etc)
  7. Save the test
  8. Now, to run the test, from menu bar select Run->Start for running a test.
  9. Run->Stop for stopping test
  10. The results will be populated in the Listeners that we have added to the test.


Test Description Test Status Remarks
Using Apache JMeter Tool,testing Broker with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of Broker to Jmeter tool and run it with users simulation of 10,50,100,150,200.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body><tns:getAllGroups xmlns:tns="http://www.eclipse.org/cosmos/broker" /> </soapenv:Body></soapenv:Envelope>

Expected Result:

  1. Broker requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing StatDataManager with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of StatDataManager to Jmeter tool and run it with users simulation of 10,50,100,150,200.

Expected Result:

  1. StatDataManager requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing LoggingDataManager with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of LoggingDataManager to Jmeter tool and run it with users simulation of 10,50,100,150,200.

Expected Result:

  1. LoggingDataManager requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing FederatingCMDB with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP requests of FederatingCMDB to Jmeter tool and run it with users simulation of 10,50,100,150,200.


<cmdbf:itemTemplate id="all-items"> </cmdbf:itemTemplate> <cmdbf:relationshipTemplate id="all-relationships"> <cmdbf:sourceTemplate ref="all-items" /> <cmdbf:targetTemplate ref="all-items" /> </cmdbf:relationshipTemplate> </cmdbf:query></soapenv:Body></soapenv:Envelope>


<cmdbf:mdrId /> <cmdbf:itemList> <cmdbf:item> <cmdbf:record xmlns="http://school">

<student>

<identity firstName="Mike" id="03" lastName="Lee"> </identity>

</student> <cmdbf:recordMetadata> <cmdbf:recordId>03</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>03</cmdbf:localId> </cmdbf:instanceId> </cmdbf:item> <cmdbf:item> <cmdbf:record xmlns="http://school">

<student>

<identity firstName="Jane" id="02" lastName="Ryerson"> </identity>

</student> <cmdbf:recordMetadata> <cmdbf:recordId>02</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>02</cmdbf:localId> </cmdbf:instanceId> </cmdbf:item> <cmdbf:item> <cmdbf:record xmlns="http://school">

<student>

<identity firstName="Bob" id="01" lastName="Davidson"> </identity>

</student> <cmdbf:recordMetadata> <cmdbf:recordId>01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:item> <cmdbf:item> <cmdbf:record xmlns="http://school">

<teacher>

<identity firstName="Heather" id="staff02" lastName="Reeba"> </identity>

</teacher> <cmdbf:recordMetadata> <cmdbf:recordId>staff02</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff02</cmdbf:localId> </cmdbf:instanceId> </cmdbf:item> <cmdbf:item> <cmdbf:record xmlns="http://school">

<teacher>

<identity firstName="Dawn" id="staff01" lastName="Johnson"> </identity>

</teacher> <cmdbf:recordMetadata> <cmdbf:recordId>staff01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:item> </cmdbf:itemList> <cmdbf:relationshipList> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff02</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>03</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="PHY01" name="Physics">

<students>

<enrolledStudent idRef="02"> </enrolledStudent>

<enrolledStudent idRef="03"> </enrolledStudent>

</students>

<teacher idRef="staff02"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>PHY01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>PHY01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff02</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>02</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="MAT01" name="Mathematics">

<students>

<enrolledStudent idRef="01"> </enrolledStudent>

<enrolledStudent idRef="02"> </enrolledStudent>

</students>

<teacher idRef="staff02"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>MAT01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>MAT01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff02</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>02</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="PHY01" name="Physics">

<students>

<enrolledStudent idRef="02"> </enrolledStudent>

<enrolledStudent idRef="03"> </enrolledStudent>

</students>

<teacher idRef="staff02"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>PHY01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>PHY01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff02</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>01</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="MAT01" name="Mathematics">

<students>

<enrolledStudent idRef="01"> </enrolledStudent>

<enrolledStudent idRef="02"> </enrolledStudent>

</students>

<teacher idRef="staff02"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>MAT01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>MAT01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff01</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>03</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="ECM01" name="Economics">

<students>

<enrolledStudent idRef="01"> </enrolledStudent>

<enrolledStudent idRef="03"> </enrolledStudent>

</students>

<teacher idRef="staff01"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>ECM01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>ECM01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> <cmdbf:relationship> <cmdbf:source> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>staff01</cmdbf:localId> </cmdbf:source> <cmdbf:target> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>01</cmdbf:localId> </cmdbf:target> <cmdbf:record xmlns="http://school">

<class courseCode="ECM01" name="Economics">

<students>

<enrolledStudent idRef="01"> </enrolledStudent>

<enrolledStudent idRef="03"> </enrolledStudent>

</students>

<teacher idRef="staff01"> </teacher>

</class> <cmdbf:recordMetadata> <cmdbf:recordId>ECM01</cmdbf:recordId> </cmdbf:recordMetadata> </cmdbf:record> <cmdbf:instanceId> <cmdbf:mdrId>org.eclipse.cosmos.samples.cmdbf.XMLRepository</cmdbf:mdrId> <cmdbf:localId>ECM01</cmdbf:localId> </cmdbf:instanceId> </cmdbf:relationship> </cmdbf:relationshipList> </cmdbf:registerRequest></soapenv:Body></soapenv:Envelope>

Expected Result:

  1. FederatingCMDB requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing ExampleMDR with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of ExampleMDR to Jmeter tool and run it with users simulation of 10,50,100,150,200.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel"> <st:itemTemplate id="items" suppressFromResult="false"> <st:recordConstraint> <st:recordType localName="student" namespace="http://schemas.examplemdr.org/schema/ExampleMDR" /> </st:recordConstraint></st:itemTemplate></st:query> </soapenv:Body> </soapenv:Envelope>

Expected Result:

  1. ExampleMDR requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing SMLMDR with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of SMLMDR to Jmeter tool and run it with user samples of 10,50,100,150,200

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel"> <st:itemTemplate id="cableTemplate"> <st:recordConstraint> <st:propertyValue localName="format" namespace="propertyModel"> <st:equal>RJ45</st:equal> </st:propertyValue> </st:recordConstraint> </st:itemTemplate> </st:query> </soapenv:Body> </soapenv:Envelope>

Expected Result:

  1. SMLMDR requests with all above user samples should work fine with appropriate turn around time
 Passed None
Using Apache JMeter Tool,testing AperiMdr with samples of 10,50,100,150,200 users simulation
  1. Follow the procedure to create the test, running the test using JMeter.
  2. Provide below SOAP request of AperiMdr to Jmeter tool and run it with 10,50,100,150,200 users simulation

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel"> <st:itemTemplate id="items" suppressFromResult="false"> <st:recordConstraint> <st:recordType localName="StoragePool" namespace="http://schemas.aperimdr.org/schema/AperiMDR" /> </st:recordConstraint> </st:itemTemplate> </st:query> </soapenv:Body> </soapenv:Envelope>

Expected Result:

  1. AperiMdr requests with all above user samples should work fine with appropriate turn around time
 Passed None


Scalability/Stability Tests

Test Description Test Status Remarks
Using Apache JMeter Tool,testing ExampleMDR with multiple volumes of test data (data.xml) 1MB,

2MB,2.5MB with user samples of 10,50,100,150,200.

  1. Modify data.xml file to insert test data with 5000,10000,15000 student records
  2. Follow the procedure to create the test, running the test using JMeter.
  3. Provide below SOAP request of ExampleMDR to Jmeter tool and run it with 10,50,100,150,200 users simulation

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"> <soapenv:Body> <st:query xmlns:st="http://cmdbf.org/schema/1-0-0/datamodel"> <st:itemTemplate id="items" suppressFromResult="false"> <st:recordConstraint> <st:recordType localName="student" namespace="http://schemas.examplemdr.org/schema/ExampleMDR" /> </st:recordConstraint></st:itemTemplate></st:query> </soapenv:Body> </soapenv:Envelope>

Expected Result:

  1. ExampleMDR requests with all above user samples should work fine with appropriate turn around time
 Passed Found "Java Heap Space" and "Post too Large" configuration issues while testing with 2MB data.mxl file

QA Test Results

  • DataVisualization.UI.Component.Widget_1209468870674.execution

http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.cosmos/tests/common/org.eclipse.cosmos.examples.e2e.tests/manual-results/DataVisualization.UI.Component.Widget_1209468870674.execution?root=Technology_Project&view=log

http://wiki.eclipse.org/images/9/90/Cosmos_i10_performancetestResults.zip

Retrieved from "https://wiki.eclipse.org/index.php?title=COSMOS_i10_QA_Phase_Execution&oldid=96352"

This page was last modified 07:03, 7 May 2008 by Srinivas Reddy.

Back to the top