COSMOS i10 QA Phase Execution

This page is created by QA to track all i10 test execution activities from 21 Apr'08 to 02 May'08. This page will be filled in by the QA team as i10 testing phase proceeds.

QA Test Execution Activities for i10

Primary Tasks:

1. E2E testing on Windows XP, 2003 and Red Hat Enterprise Linux Server 5.0 (http://wiki.eclipse.org/COSMOS_DEMO_i10 )
2. i10 ERs – JUnits/Manual Tests inspection (http://wiki.eclipse.org/Cosmos1.0Features#Iteration_10_Enhancements)
3. Data Visualization TPTP Manual Tests Execution (DataVisualization.UI.Component.Widget.testsuite / UI.Reports.testsuite)

Secondary Tasks:

1. Execution of i10 supplement e2e tests (negative) from QA.
2. Basic operational efficiency tests (limited performance tests)

For complete i10 QA Activities, refer http://wiki.eclipse.org/COSMOS_QA_i10_Activities

E2E testing status (WinXP SP2/Win2K3 SP1/RHES 5.0)

Build Installation and Configuration

Test Description	Test Status Windows XP SP2	Test Status Windows 2003 SP1	Test Status RHES 5.0	Remarks
<cosmos-demo>/bin/configDemo.bat <tomcatDir> <cosmosDir> <birt-path> <dojo-path> <axis2war-path>	Passed	Passed	Passed	None

Starting COSMOS components : Management Console

Test Description	Test Status Windows XP SP2	Test Status Windows 2003 SP1	Test Status RHES 5.0	Remarks
<cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:8080/axis2/services/Broker	Passed	Passed	Passed	None
COSMOS> broker registerAllDataManagers	Passed	Passed	Passed	Found warnings - "log4j:WARN No appenders could be found for logger (org.apache.axis2.util.Loader). log4j:WARN Please initialize the log4j system properly."
COSMOS> broker getDataManagers	Passed	Passed	Passed	None
COSMOS> finddatamanagers http://localhost:8080/axis2/services/CosmosServiceFinder	Passed	Passed	Passed	None
COSMOS> broker deregister domsr01-xp StatDataManager	Passed	Passed	Passed	None
COSMOS> broker deregister domsr01-xp LoggingDataManager	Passed	Passed	Passed	None
COSMOS> broker deregister domsr01-xp TestFederatingCMDB	Passed	Passed	Passed	None
COSMOS> broker deregister domsr01-xp ExampleMdr	Passed	Passed	Passed	None
COSMOS> broker deregister domsr01-xp SMLMDR	Passed	Passed	Passed	None
COSMOS> broker deregister domsr01-xp AperiMdr	Passed	Passed	Passed	None
COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt	Passed	Passed	Passed	None
COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testPropertyQuery.txt	Passed	Passed	Passed	None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-students.txt	Passed	Passed	Passed	None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\all-teachers.txt	Passed	Passed	Passed	None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\relationship-all-classes.txt	Passed	Passed	Passed	None
COSMOS> graphQuery domsr01-xp ExampleMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\ExampleMDR\teaches-relationship.txt	Passed	Passed	Passed	None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt	Passed	Passed	Passed	None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_localid.txt	Passed	Passed	Passed	None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_display_name.txt	Passed	Passed	Passed	None
COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_relationship.txt	Passed	Passed	Passed	None
COSMOS> COSMOS> graphQuery domsr01-xp TestFederatingCMDB D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\TestFederatingCMDB\all_items.txt	Passed	Passed	Passed	Empty Response, no data found.

Using the COSMOS Web User Interface

Test Description	Test Status Windows XP SP2	Test Status Windows 2003 SP1	Test Status RHES 5.0	Remarks
http://localhost:8080/COSMOSUI/?page=cosmos	Passed	Passed	Passed	Found All 6 DataManagers under COSMOS Web UI
StatDataManager server 33 tomcat1181829221515 CBE	Passed	Passed	Passed	Statistical Report generated successfully
StatDataManager server 33 tomcat1181829221515 TomcatSet	Passed	Passed	Passed	Statistical Report generated successfully
LoggingDataManager access_log.cbe.xml	Passed	Passed	Passed	Top 10 Log Report generated successfully
LoggingDataManager windows_app_log.cbe.xml	Passed	Passed	Passed	Top 10 Log Report generated successfully
Sample Repository with Registration Submit Query	Passed	Passed	Passed	Query for all-items resulted Empty Reponse, no data found.
Sample Repository with Registration Register Configuration Items...	Passed	Passed	Passed	Registration with Example MDR is successful
Sample Repository with Registration Deregister Configuration Items...	Passed	Passed	Passed	Deregistration with Example MDR is successful
Sample Repository with Registration Submit Query	Passed	Passed	Passed	Query for all-items resulted Empty Reponse, no data found.
Sample Repository with Registration View MetaData	Passed	Passed	Passed	No Metadata available
Resource Repository Submit Query	Passed	Passed	Passed	Queries executed successfully with proper results
Resource Repository View MetaData	Passed	Passed	Passed	No Metadata available
AperiMDR Submit Query	Passed	Passed	Passed	Using Query builder able to execute and validate queries properly
AperiMDR View MetaData	Passed	Passed	Passed	Displayed MetaData properly
Sample Repository with Query Get All Students	Passed	Passed	Passed	Displayed Query Result for Students data correctly
Sample Repository with Query Get All Teachers	Passed	Passed	Passed	Displayed Query Result for Teachers data correctly
Sample Repository with Query Get All Classes	Passed	Passed	Passed	Displayed Query Result for Classes data correctly
Sample Repository with Query Staff01	Passed	Passed	Passed	Displayed data for 'students taught by teacher-staff01' correctly
Sample Repository with Query Staff02	Passed	Passed	Passed	Displayed data for 'students taught by teacher-staff02' correctly
Sample Repository with Query Submit Query	Passed	Passed	Passed	Using Query Builder, able to create and execute queries correctly
Sample Repository with Query View MetaData	Passed	Passed	Passed	Displayed MetaData information correctly

Data Visualization TPTP Manual Tests

/org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Component.Widget.testsuite

Test Description	Test Status Windows XP SP2	Test Status RHES 5.0	Remarks
navigate	Passed	Passed	6 nodes should be shown under "Data Managers" No "Monitoring Data Repository" node, it should be "StatDataManager" node
navigate2	Passed	Passed	No "Create Query" found, it should be "Submit Query"
properties	Passed	Passed	Given EPR is wrong, it should be http://localhost:8080/axis2/services/ExampleMdrDataManager
visualize log report	Passed	Passed	No "Monitoring Data Repository" available, it should be "StatDataManager
visualize stat report	Passed	Passed	None
properties 2	Passed	Passed	None
example cmdbf query	Passed	Passed	XML output contents seems to be wrong, actual result shows only one student detail - "firstName="Mike" id="03 lastname="Lee""
SML CMDBF Graph Response Viewer	Passed	Passed	None
Example MDR CMDBF Graph Response Viewer	Passed	Passed	No cancel button available under details view window, it should be OK button.
Multiple Graph Response Viewers	Passed	Passed	No "Cancel" button available under detail area, it should be Ok button
Registration Example	Passed	Passed	None
DeRegistration Example	Passed	Passed	None
Partial Registration of MDR with CMDBf	Passed	Passed	None
Partial DeRegistration of MDR with CMDBf	Passed	Passed	None

/org.eclipse.cosmos.examples.e2e.tests/manual/DataVisualization.UI.Reports.testsuite

Test Description	Test Status Windows XP SP2	Test Status RHES 5.0	Remarks
LogReport	Passed	Passed	None
StatReport	Passed	Passed	None

i10 ERs - JUnits/Manual Tests Inspection

Bugzilla ID	ID Summary	JUnits Status	Manual Tests	Remarks

i10 End2End Supplement Tests (Negative)

COSMOS Command Line Client : Management Console

Test Description	Test Status Windows XP SP2	Test Status RHES 5.0	Remarks
Configure tomcat port to 9080 (other than default port : 8080) and test the following: <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker Expected Result: Should diplay COSMOS prompt	Passed	Passed	None
Test Broker registration other than defaul port 8080 (say 9080): <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> broker registerAllDataManagers http://localhost:9080/axis2/services/CosmosServiceFinder Expected Result: All DataManagers should successfuly get registered to broker on port 9080	Failed	Failed	DataManagers are getting registered on default port 8080 only, ignored actual port 9080
Without broker registration, test the following: <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> broker getDataManagers Expected Result: Should display empty list of DataManagers	Passed	Passed	None
After succesful broker registration, test the following: <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> broker getDataManagers Expected Result: Should display all 6 DataManagers registered with broker	Passed	Passed	None
Without any broker registration, test the following <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> finddatamanagers http://localhost:9080/axis2/services/CosmosServiceFinder Expected Result: Should display all 6 DataManagers deployed under tomcat with port 9080	Passed	Passed	None
Test deregistration of StatDataManager with valid hostname and valid datamanager name : <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> broker deregister domsr01-xp StatDataManager Expected Result: Deregistration should be successful	Passed	Passed	None
Test deregistration of StatDataManager with invalid hostname and valid datamanager name : <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> broker deregister unknown StatDataManager Expected Result: Deregistration should fail and should display appropriate error message	Failed	Failed	No Failure message, it still shows "DataManager deregistered"
Test deregistration of StatDataManager with valid hostname and invalid datamanager name : <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> broker deregister domsr01-xp StatDataMgr Expected Result: Deregistration should fail and should display appropriate error message	Failed	Failed	No Failure message, it still shows "DataManager deregistered"
Test graphQuery with valid hostname,valid DataManager,valid query path: <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt Expected Result: Query execution should be succesful.	Passed	Passed	This is depended on Broker registration issue, fixed manually to test this.
Test graphQuery with invalid hostname,valid DataManager,valid query path: <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> graphQuery unknown SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt Expected Result: Query execution should fail and throw appropriate error message.	Passed	Passed	Displaying error message as "SMLMDR is not registered at the broker."
Test graphQuery with valid hostname,invalid DataManager,valid query path: <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> graphQuery domsr01-xp StatDataManager D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt Expected Result: Query execution should fail and throw appropriate error message.	Passed	Passed	Displayed error message as "StatDataManager is not registered at the broker."
Test graphQuery with valid hostname,valid DataManager,invalid query path: <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> graphQuery domsr01-xp SMLMDR D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDRTest\testInstanceIdQuery.txt Expected Result: Query execution should fail and throw appropriate error message.	Passed	Passed	Displayed "java.io.FileNotFoundException:" error message tree
Test graphQuery with valid hostname,valid DataManager,valid query path with spaces: <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> graphQuery domsr01-xp SMLMDR "D:\cosmos i10\cosmos-demo\cosmos-client\cmdbfQuery\SMLMDR\testInstanceIdQuery.txt" Expected Result: Query execution should be successful	Failed	Failed	Displayed "java.io.FileNotFoundException:" error message tree
Test graphQuery for AperMdr with valid data, with invalid Aperi database settings: <cosmos-demo>\cosmos-client\cosmosClient.bat http://localhost:9080/axis2/services/Broker COSMOS> graphQuery domsr01-xp AperiMdr D:\cosmosi10\cosmos-demo\cosmos-client\cmdbfQuery\AperiMdr\StorageSystem_all_items.txt Expected Result: Query execution should fail, with DB Connection error	Passed	Passed	Displayed "aperiDB.getStorageSystems error"

Using the COSMOS Web User Interface

Test Description	Test Status Windows XP SP2	Test Status RHES 5.0	Remarks
http://localhost:8080/COSMOSUI/?page=cosmos	Passed	Passed	Found All 6 DataManagers under COSMOS Web UI
StatDataManager server 33 tomcat1181829221515 CBE	Passed	Passed	Statistical Report generated successfully
StatDataManager server 33 tomcat1181829221515 TomcatSet	Passed	Passed	Statistical Report generated successfully
LoggingDataManager access_log.cbe.xml	Passed	Passed	Top 10 Log Report generated successfully
LoggingDataManager windows_app_log.cbe.xml	Passed	Passed	Top 10 Log Report generated successfully
Sample Repository with Registration Submit Query	Passed	Passed	Query for all-items resulted Empty Reponse, no data found.
Sample Repository with Registration Register Configuration Items...	Passed	Passed	Registration with Example MDR is successful
Sample Repository with Registration Deregister Configuration Items...	Passed	Passed	Deregistration with Example MDR is successful
Sample Repository with Registration Submit Query	Passed	Passed	Query for all-items resulted Empty Reponse, no data found.
Sample Repository with Registration View MetaData	Passed	Passed	No Metadata available
Resource Repository Submit Query	Passed	Passed	Queries executed successfully with proper results
Resource Repository View MetaData	Passed	Passed	No Metadata available
AperiMDR Submit Query	Passed	Passed	Using Query builder able to execute and validate queries properly
AperiMDR View MetaData	Passed	Passed	Displayed MetaData properly
Sample Repository with Query Get All Students	Passed	Passed	Displayed Query Result for Students data correctly
Sample Repository with Query Get All Teachers	Passed	Passed	Displayed Query Result for Teachers data correctly
Sample Repository with Query Get All Classes	Passed	Passed	Displayed Query Result for Classes data correctly
Sample Repository with Query Staff01	Passed	Passed	Displayed data for 'students taught by teacher-staff01' correctly
Sample Repository with Query Staff02	Passed	Passed	Displayed data for 'students taught by teacher-staff02' correctly
Sample Repository with Query Submit Query	Passed	Passed	Using Query Builder, able to create and execute queries correctly
Sample Repository with Query View MetaData	Passed	Passed	Displayed MetaData information correctly

Basic Operational efficiency tests

Test Description	Output	Status	Remarks