Difference between revisions of "TPTP-Test-Automation-20071203"

Revision as of 09:46, 3 December 2007

Attendees

• Present:
  • Paul
  • Kiryl
  • Joanna
  • Jonathan
  • Joel
  • Alan

Deliverables

• Testing Process:
  • Alan integrated the review comments and Paul posted the second draft of the Testing Process (part 1) document for review.
    • Allan will integrate review comments for the second draft of the Testing Process (part 1) document by Monday, December 3.
  • Alan will add more detailed content to the first draft of the Testing Process (part 2) by Wednesday, November 28.
  • Please review the latest drafts for both Testing Process documents for Monday, December 3.

• Common Test Infrastructure:
  • Completed:
    • CVS check-out of all of the test plug-ins in the /test-results/<project>/* modules.
    • Configure and run the Agent Controller on the reference platform.
    • Paul created the root-level test suites for BVTs. The test infrastructure will execute the org.eclipse.hyades.tests/AllBVT<test type>Tests.testsuite test suites. We do not need to check-in the execution results to CVS due to disk space limitations and polluting of our test pass results. Each developer can rerun the automated tests to reproduce a failure.
    • Paul created the BVT deployment to the reference platform.
    • Automatically generating the Test Report based on the existing test execution results in CVS.
    • Paul modified the BIRT tabular test report to walk an execution result to generate a detailed test report from a root-level execution result.
  • Outstanding:
    • Check for Intel build to complete (every x hours) before launching the BVTs.
    • Automatically posting the Test Report to /tptp/test/reports/<release>.
    • Cleanup of the reference platform.
    • Documenting the test infrastructure for use in the second testing process document, which will be required by TPTP adopters or extenders.
    • Email notifications to component leads for failing test suites.
  • Issues:

1. Defect 200194: Targeted to i5.
2. Defect 190453: Targeted to i6.
3. Defect 210289: Targeted to i6.
4. Defect 210292: Targeted to i6.
5. Defect 210955: DTP has resolved the issue, available in the December 1st build. Paul has verified the patch but the symptoms still exist due to defect 211381. Both defects are now resolved and Paul will verify the fixes.
6. Defect 211476: Paul will fix in i4.

• Metrics:
  • The monitoring project lead has replied to a request to estimate the current testing costs.
    • Joanna and Guru are gathering their estimates.

Action Items

• All:
  • Review the latest drafts for both Testing Process documents for Monday, December 3.

• Alan:
  • Integrate review comments for the second draft of the Testing Process (part 1) document by Monday, December 3.
  • Add more detailed content to the first draft of the Testing Process (part 2) by Monday, December 3.

• Joel:
  • Outstanding issues for the PoC (integrate the ASF with the TPTP builds to check-out a 'Hello World' TPTP JUnit test suite from CVS, run the test suite on an Eclipse instance running on the target platform, and generate a BIRT report on the test execution using existing sample code).

• Paul:
  • Determine labor cost (PW) for a) full test pass and b) smoke test pass from project leads.
  • Encourage Project Leads to automate existing manual test suites.

Upcoming Meeting

• We will meet next on Monday, December 3 (9 AM ET).

Upcoming Discussion

• Test framework reuse:
  • See defect 171827.
  • Existing test frameworks include:
    • Test Server for the Agent Controller/Profiler
    • Test Server for the JVMTI Profiler.
  • Proposed test frameworks include:
    • Test Dashboard (see enhancement 112928 and Test Dashboard Plug-in).
    • Test automation service (see defect 87285)