Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Talk:Aperi/Test/Test Plans/IDVT"
| Line 7: | Line 7: | ||
Chris Rich | Chris Rich | ||
| + | |||
| Line 16: | Line 17: | ||
Hans Lin | Hans Lin | ||
| + | |||
| Line 24: | Line 26: | ||
This looks good, although I have some comments that you should respond to before I can approve it. | This looks good, although I have some comments that you should respond to before I can approve it. | ||
| − | - You should assume that the 'new' GUI will be built as part of the nightly build, and installable in the usual way, in time for your testing to begin. | + | - You should assume that the 'new' GUI will be built as part of the nightly build, and installable in the usual way, in time for your testing to begin. |
| + | |||
- I think section 1.7 should be deleted -- or I don't understand why it's there and empty. The information I thought would be there is in section 3.1.1. | - I think section 1.7 should be deleted -- or I don't understand why it's there and empty. The information I thought would be there is in section 3.1.1. | ||
| + | |||
- Specific references to IBM and TPC should be removed (e.g. "Source: TPC 3.1 IDVT Plan") | - Specific references to IBM and TPC should be removed (e.g. "Source: TPC 3.1 IDVT Plan") | ||
| + | |||
- We can go straight to the external wiki for this, no reason to use the internal one (once the IBM/TPC references are removed). | - We can go straight to the external wiki for this, no reason to use the internal one (once the IBM/TPC references are removed). | ||
| + | |||
- On that note, you should remove the existing title page and document control information (everything up to the table of contents) and replace it with a new title page -- no references to Tivoli, IBM Confidential, or anything like that. For an example see http://www.eclipse.org/aperi/tech/Aperi_System_Design.doc | - On that note, you should remove the existing title page and document control information (everything up to the table of contents) and replace it with a new title page -- no references to Tivoli, IBM Confidential, or anything like that. For an example see http://www.eclipse.org/aperi/tech/Aperi_System_Design.doc | ||
| + | |||
- It would be swell if this wasn't MS Word. If it stays MS Word, then I don't know how to put it on the wiki portion of our eclipse site -- but I can show you a directory I created elsewhere (not in the wiki) you can use to check it in. You are a committer, right? | - It would be swell if this wasn't MS Word. If it stays MS Word, then I don't know how to put it on the wiki portion of our eclipse site -- but I can show you a directory I created elsewhere (not in the wiki) you can use to check it in. You are a committer, right? | ||
| Line 38: | Line 45: | ||
Hardware | Hardware | ||
Two Windows servers to host APERI server components | Two Windows servers to host APERI server components | ||
| − | Why do we need two Windows machines to host APERI server components? ( One for Data and One for Device ) ? | + | -Why do we need two Windows machines to host APERI server components? ( One for Data and One for Device ) ? |
| + | |||
One Windows machine running the Data agent (and not running the server) | One Windows machine running the Data agent (and not running the server) | ||
| − | I think it will make more sense to have two windows machines for agents ( one with HBA and one without HBA to host Fabric agent and Data agent). | + | -I think it will make more sense to have two windows machines for agents ( one with HBA and one without HBA to host Fabric agent and Data agent). |
At least, we need a windows machine ( with HBA ) to have both data and fabric agents installed | At least, we need a windows machine ( with HBA ) to have both data and fabric agents installed | ||
| + | |||
One Linux machine running APERI server and agent components | One Linux machine running APERI server and agent components | ||
One NAS server | One NAS server | ||
| Line 56: | Line 65: | ||
Source: Initial Contribution | Source: Initial Contribution | ||
IDVT installation and configuration for the Initial Donation will be done using the Aperi build procedure and will be used to verify that the Data Server, Device Server, Data Agent, database, and GUI are all configured and started without errors. | IDVT installation and configuration for the Initial Donation will be done using the Aperi build procedure and will be used to verify that the Data Server, Device Server, Data Agent, database, and GUI are all configured and started without errors. | ||
| − | We need to include Fabric agent | + | |
| + | -We need to include Fabric agent | ||
Fabric Alerts and Alert Logs | Fabric Alerts and Alert Logs | ||
| − | We didn't have Disk Alerts testcases for our Phase I IDVT, I think we should do it for our phase II IDVT | + | |
| + | -We didn't have Disk Alerts testcases for our Phase I IDVT, I think we should do it for our phase II IDVT | ||
Specific Views | Specific Views | ||
Include testing of one example of each kind of view: | Include testing of one example of each kind of view: | ||
| Line 69: | Line 80: | ||
Computer L2 | Computer L2 | ||
| − | We should include the entire Topology views as follows | + | -We should include the entire Topology views as follows |
L0:Computers L1:Computers-status L2:Computer-computer | L0:Computers L1:Computers-status L2:Computer-computer | ||
L0:Fabrics L1:Fabric-fabric L2:Switch-switch | L0:Fabrics L1:Fabric-fabric L2:Switch-switch | ||
| Line 79: | Line 90: | ||
'''Tom Guinane:''' | '''Tom Guinane:''' | ||
| + | |||
| + | Front page - need to remove 'Tivoli', 'IBM', 'Confidential', etc. In fact, do a scan and get rid of anything like that in the entire document. For this document, we are Eclipse and specifically, the Aperi Storage Management Project. | ||
| + | |||
| + | page 2 - If you did this as a wiki page, you could remove the document control information from here and only list in the 'discussion' section. You could also reference the document source directly (the URL for the page) | ||
| + | |||
| + | page 4, Introduction - Delete the words "and before coding begins for Phase 3" out of the second sentence. It is unnecessary and is not what will really happen. | ||
| + | |||
| + | page 5, Section 1.3, last sentence - we will be using Bugzilla at Eclipse, not CMVC to track defects. I'm not sure how we tell which defects are ours in Bugzilla. There must be some code similar to CMVC. We need to check that out. | ||
| + | |||
| + | page 5, Section 1.4 - need to add "The IDVT Test Plan has been approved" as the first entry criteria for the first part of IDVT. | ||
| + | |||
| + | page 5, Section 1.4, part two - need to fix the wording of the second bullet. | ||
| + | |||
| + | page 5, Section 1.5, bullet two - Any particular reason we are deviating from our normal 95% success rate with no sev 1's or 2's? This leaves us wide open for a success rate. | ||
| + | |||
| + | page 5, Section 1.4, bullet three - To exit test, we need to have objective measurement evidence. How do you plan to measure that for this item? Just the existence of it? What about the quality of this item? This might be a difficult to validate. | ||
| + | |||
| + | page 7, 2.1.1 - I like this. Just remember, as it is a test entry criteria, you will need to provide evidence for each of these. | ||
| + | |||
| + | page 8, 2.1.2 - performance numbers at the top - should these be listed in this external document? | ||
| + | |||
| + | page 9, 3.1.4, GUI - this is good. We will need to have the plan amended for the GUI tests before that testing begins. | ||
| + | |||
| + | page 11, 4.1 - I think the external wiki is fine for this. Will we have S-curves (like we did in Phase 1)? | ||
| + | |||
| + | page 12, 5.1.3 - GUI date will probably move out to 3/16/07 with the GUI changes. | ||
| + | |||
| + | Section 5, most pages - remove TPC references from Source areas | ||
| + | |||
| + | |||
| + | '''Chris King:''' | ||
| + | |||
| + | I looked over the plan and have some minor suggestions. | ||
| + | |||
| + | You might want to add test cases for the online help in the new Eclipse RCP-based user interface. This interface will have an accompanying online help system using the new Eclipse framework and should work like the existing help system in legacy user interface (i.e., press F1 to access the help). | ||
| + | |||
| + | For example: | ||
| + | |||
| + | - Ensure that the correct help topic is appearing for the product window that invoked the help. For example, press F1 from one of the generated BIRT reports (Disk Capacity > By Disk) included in the new interface and press F1 to confirm the correct help topic appears. | ||
| + | |||
| + | - Check the links between the entries in the help's table of contents and the actual topics in the help. For example, click on an entry in the help's table of contents (left pane) and confirm that the corresponding help topic appears (in the right pane). | ||
| + | |||
| + | - Verify links from one help topic to another. For example, confirm that the link in the Disk Capacity > By Disk help window correctly links to the help topic that contains overview help for all reports. | ||
| + | |||
| + | - Test the Back, Forward, and print buttons in the tool/menu bar. | ||
| + | |||
| + | - Test the cheat sheets (if any): Ensure that the correct cheat sheet appears for the product window, check any links within the cheat sheet to confirm that they link correctly, etc. | ||
Revision as of 17:52, 24 October 2006
Aperi IDVT Test Plan - Phase 2
Document Approvers (Please add "I Approve" followed by your signature below your name to indicate your approval of the test plan)
Ted Slupesky
Chris Rich
Mandatory Document Reviewers
Dave Wolfe
Tom Guinane
Hans Lin
Comments
Ted Slupesky:
This looks good, although I have some comments that you should respond to before I can approve it.
- You should assume that the 'new' GUI will be built as part of the nightly build, and installable in the usual way, in time for your testing to begin.
- I think section 1.7 should be deleted -- or I don't understand why it's there and empty. The information I thought would be there is in section 3.1.1.
- Specific references to IBM and TPC should be removed (e.g. "Source: TPC 3.1 IDVT Plan")
- We can go straight to the external wiki for this, no reason to use the internal one (once the IBM/TPC references are removed).
- On that note, you should remove the existing title page and document control information (everything up to the table of contents) and replace it with a new title page -- no references to Tivoli, IBM Confidential, or anything like that. For an example see http://www.eclipse.org/aperi/tech/Aperi_System_Design.doc
- It would be swell if this wasn't MS Word. If it stays MS Word, then I don't know how to put it on the wiki portion of our eclipse site -- but I can show you a directory I created elsewhere (not in the wiki) you can use to check it in. You are a committer, right?
Hans Lin:
Environmental Requirements This section lists both the necessary and desired properties required for the test environment. Hardware Two Windows servers to host APERI server components -Why do we need two Windows machines to host APERI server components? ( One for Data and One for Device ) ?
One Windows machine running the Data agent (and not running the server) -I think it will make more sense to have two windows machines for agents ( one with HBA and one without HBA to host Fabric agent and Data agent). At least, we need a windows machine ( with HBA ) to have both data and fabric agents installed
One Linux machine running APERI server and agent components One NAS server One tape device At least one subsystem visible to test environment Brocade switch McData switch Cisco switch A large number of storage subsystems (desired for performance testing) Storage subsystems with a large number of volumes (desired for performance testing) A large number of hosts (desired for performance testing)
Install and Configuration Tests Source: Initial Contribution IDVT installation and configuration for the Initial Donation will be done using the Aperi build procedure and will be used to verify that the Data Server, Device Server, Data Agent, database, and GUI are all configured and started without errors.
-We need to include Fabric agent
Fabric Alerts and Alert Logs
-We didn't have Disk Alerts testcases for our Phase I IDVT, I think we should do it for our phase II IDVT Specific Views Include testing of one example of each kind of view: SAN L0 SAN L1 SAN L2 Computer L0 Computer L1 Computer L2
-We should include the entire Topology views as follows L0:Computers L1:Computers-status L2:Computer-computer L0:Fabrics L1:Fabric-fabric L2:Switch-switch L0:Storage L1:Subsystems-status L2:Subsystem-subsystem L0:Other L1:Others-status L2:Other-other
In our Phase I donation, we didn't include Fabric agent support so there is no zoning support for Cisco and McData switches. We should have some zone test cases for out Phase II IDVT
Tom Guinane:
Front page - need to remove 'Tivoli', 'IBM', 'Confidential', etc. In fact, do a scan and get rid of anything like that in the entire document. For this document, we are Eclipse and specifically, the Aperi Storage Management Project.
page 2 - If you did this as a wiki page, you could remove the document control information from here and only list in the 'discussion' section. You could also reference the document source directly (the URL for the page)
page 4, Introduction - Delete the words "and before coding begins for Phase 3" out of the second sentence. It is unnecessary and is not what will really happen.
page 5, Section 1.3, last sentence - we will be using Bugzilla at Eclipse, not CMVC to track defects. I'm not sure how we tell which defects are ours in Bugzilla. There must be some code similar to CMVC. We need to check that out.
page 5, Section 1.4 - need to add "The IDVT Test Plan has been approved" as the first entry criteria for the first part of IDVT.
page 5, Section 1.4, part two - need to fix the wording of the second bullet.
page 5, Section 1.5, bullet two - Any particular reason we are deviating from our normal 95% success rate with no sev 1's or 2's? This leaves us wide open for a success rate.
page 5, Section 1.4, bullet three - To exit test, we need to have objective measurement evidence. How do you plan to measure that for this item? Just the existence of it? What about the quality of this item? This might be a difficult to validate.
page 7, 2.1.1 - I like this. Just remember, as it is a test entry criteria, you will need to provide evidence for each of these.
page 8, 2.1.2 - performance numbers at the top - should these be listed in this external document?
page 9, 3.1.4, GUI - this is good. We will need to have the plan amended for the GUI tests before that testing begins.
page 11, 4.1 - I think the external wiki is fine for this. Will we have S-curves (like we did in Phase 1)?
page 12, 5.1.3 - GUI date will probably move out to 3/16/07 with the GUI changes.
Section 5, most pages - remove TPC references from Source areas
Chris King:
I looked over the plan and have some minor suggestions.
You might want to add test cases for the online help in the new Eclipse RCP-based user interface. This interface will have an accompanying online help system using the new Eclipse framework and should work like the existing help system in legacy user interface (i.e., press F1 to access the help).
For example:
- Ensure that the correct help topic is appearing for the product window that invoked the help. For example, press F1 from one of the generated BIRT reports (Disk Capacity > By Disk) included in the new interface and press F1 to confirm the correct help topic appears.
- Check the links between the entries in the help's table of contents and the actual topics in the help. For example, click on an entry in the help's table of contents (left pane) and confirm that the corresponding help topic appears (in the right pane).
- Verify links from one help topic to another. For example, confirm that the link in the Disk Capacity > By Disk help window correctly links to the help topic that contains overview help for all reports.
- Test the Back, Forward, and print buttons in the tool/menu bar.
- Test the cheat sheets (if any): Ensure that the correct cheat sheet appears for the product window, check any links within the cheat sheet to confirm that they link correctly, etc.