Difference between revisions of "TPTP-Test-Automation-20071203"

Revision as of 13:07, 3 December 2007

Attendees

• Present:
  • Paul
  • Jonathan
  • Joel

• Absent:
  • Kiryl - conflict
  • Joanna
  • Alan

Deliverables

• Testing Process:
  • Paul replied to Alan's note with repsonses to his questions on the review comments.
    • Alan will post the questions/answers to asreview comments.
  • Paul added review comments for the second draft of the Testing Process (part 1) document.
    • Allan will integrate review comments for the second draft of the Testing Process (part 1) document.
  • Please review the latest drafts for both Testing Process documents.
  • Defect 211777 will be managed by the Platform Project Lead to track the delivery of the final Testing Process documents (part 1 and part 2). See the 4.5 Weekly Schedule).

• Common Test Infrastructure:
  • Completed:
    • CVS check-out of all of the test plug-ins in the /test-results/<project>/* modules.
    • Configure and run the Agent Controller on the reference platform.
    • Paul created the root-level test suites for BVTs. The test infrastructure will execute the org.eclipse.hyades.tests/bvt/AllBVT<test type>Tests.testsuite test suites. We do not need to check-in the execution results to CVS due to disk space limitations and polluting of our test pass results. Each developer can rerun the automated tests to reproduce a failure.
    • Paul created the BVT deployment to the reference platform.
    • Automatically generating the Test Report based on the existing test execution results in CVS.
    • Paul modified the BIRT tabular test report to walk an execution result to generate a detailed test report from a root-level execution result.
    • Cleanup of the reference platform is done when staring the BVT.
  • Outstanding:
    • Check for Intel build to complete (every x hours) before launching the BVTs (see defect 200351).
      • Joel will run the BVT when the web site has been updated with Intel's portion of the build.
    • Automatically posting the Test Report to /tptp/test/reports/<release>.
      • An ANT task is available and needs to copied/refactored.
    • Documenting the test infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders (see defect 211752.
    • Email notifications to component leads for failing test suites.
  • Issues:

1. Defect 200194: Targeted to i5.
2. Defect 190453: Targeted to i6.
3. Defect 210289: Targeted to i6.
4. Defect 210292: Targeted to i6.
5. Defect 210955: DTP has resolved the issue, available in the December 1st build. Paul has verified the patch but the symptoms still exist due to defect 211381. Both defects are now resolved and Paul will verify the fixes.
6. Defect 211476: Paul will fix during i4 TP1.
7. Few minor issues with the executable bit on Linux not being set properly.

• • Defect 211751 will be managed by the Platform Project Lead to track the delivery of the final Common Test Infrastructure. Defect 211752 will be managed by the Platform Project Lead to track documenting the test infrastructure for use in the second testing process document. See the 4.5 Weekly Schedule).

• Metrics:
  • The monitoring project lead has replied to a request to estimate the current testing costs.
    • Joanna and Guru are gathering their estimates.

Action Items

• All:
  • Review the latest drafts for both Testing Process documents.

• Alan:
  • Defect 211777.

• Joel:
  • Defect 211751.
  • Defect 211752.

• Paul:
  • Determine labor cost (PW) for a) full test pass and b) smoke test pass from project leads.
  • Encourage Project Leads to automate existing manual test suites.

Upcoming Meeting

• Since this is the wrap-up call, the outstanding deliverables will be managed by the Platform Project Lead, under the 4.5 plan. See the 4.5 Weekly Schedule).

Upcoming Discussion

• Test framework reuse:
  • See defect 171827.
  • Existing test frameworks include:
    • Test Server for the Agent Controller.
    • Test Server for the JVMTI Profiler.
  • Proposed test frameworks include:
    • Test Dashboard (see enhancement 112928 and Test Dashboard Plug-in).
    • Test automation service (see defect 87285)