TPTP-Test-Tools-20080602

Attendees

• Present:
  • Paul Slauenwhite
  • Tony Wang
  • Jerome Bozier
  • DuWayne Morris

4.5

Release Information

• 4.5 will ship with the Ganymede Simultaneous Release (target release date: June 25, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• This week is I8 TP2 (target: June 6).
  • All defects require patch reviews, Stop Ship Template, and Project/PMC approval.

• RC1 was delivered on Tuesday, May 20.

• RC2 was delivered for Tuesday, May 27.

• RC3 is scheduled for Tuesday, June 3.

• RC4 (final RC) is scheduled for Friday, June 6.

Defect Verification

• A note has been sent to the originators of all resolved defects reminding them to verify/close these defects before Ganymede releases.

• Please verify/close your resolved defects this week:
  • Paul.
  • Tony.
  • Duwayne.
  • Jerome.

Testing

• We want to complete the I8 TP2 by Thursday June 5.

• The candidate driver will be announced by the Release Engineering Team later today.

• Please use the TPTP all-in-one package and install dependent/As-Is/Technology Preview components in the <Eclipse installation directory>/eclipse/dropins directory.
  • Note: When testing in a Java 1.4.x environment, do NOT use the TPTP all-in-one package since it requires Java 1.5.x+.

4.5 I8 TP2

• TP2 testing assignments.
  • Note, do not run any of the documentation technical review test suites (/org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.<component>.technical_review.Windows_IA32.testsuite).

News Group

• The following news group posts require an IMMEDIATE reply:
  • Duwayne:
    • Re: Using wait command in AGR script (March 11, 2008 2:07 PM)
    • Re: HTTP Recording stops immediately after start-up (FireFox) or never records data (IE) (March 25, 2008 1:15 PM)
    • tptp execuation task issue (April 2, 2008 3:12 PM)
    • Multi-Language tests with AGR (April 7, 2008 3:11 PM)
    • Testing an application with Custom component (April 8, 2008 11:03 AM)
    • Re: NoClassDefFoundError: org.eclipse.hyades.test.common...? (April 10, 2008 5:33 PM)
    • Re: Multiple instances for Verification Class (April 28, 2008 3:11 PM)
    • Re: Reusable Thread Error in standard mode (April 29, 2008 6:12 PM)

Test Automation Initiative

• Paul and the Release Engineering Team are working on consolidating our testing process and automation infrastructure:
  • For more information, see the WIKI page.
  • To see the BVT results for a specific build, open the Build Reports page for a specific build and select BVT Results.
  • Outstanding defects include:
    • Paul:
      • JUnit and JUnit Plug-in test execution add incorrect elements to the plugin.xml file.
      • AGR Tests require absolute path to Equinox JAR
      • Clear remote workspace launch does not work for remote test execution
      • AGR Tests cannot be run with start ACServer
      • Test Execution Tabular Report contains empty or incorrect child reports.
      • IExecutionDeploymentAdapter.cleanUpTestAssets no longer called
    • Duwayne:
      • Connection property does not deploy all dependencies
      • Test Execution fails when project defined incorrectly - ResourceException
      • Connection property does not allow JUnit Plug-in Tests to be executed
      • ASF marshalling return value between service and client adapter does not have support for failures / exceptions
      • Confusing exceptions when running automated services.
      • ASF Test Execution Service races with AutoBuildJob -- can lead to unexpected test results
      • ANT test results property requires trailing slash to specify location
      • Exceptions when running ASF tasks.
      • Only the name of the execution result is used from the results property.

Enhancements

• Queries.
• Schedule.

• Alexander Nyßen:
  • Providing the ability to record user's interactions with GEF objects in a GEF editor: Alexander (TPTP user and contributor) has contributed a patch and documentation for basic GEF/Draw2D support in the AGR. AG/PMC approved and Contribution Questionnaire (CQ) approved. Since the AGR is an As-Is component, Paul will complete the integration, testing, and check-in after the I8 TP2 AGR tests have been completed.

Defects

• Queries.

• Schedule.

• Defects require a reply:
  • Duwayne:
    • jdt.ui bundle activator assumes PlatformUI.getWorkbench(), causing autobuild failure when running headless
    • Cannot remove a test invocation to a removed HTTP request.
    • Unable to publish BIRT reports through ASF
    • NullPointerException when generating a Test Execution Tabular Report using ASF.

• Defects require a sizing:
  • Duwayne:
    • Test Execution fails when project defined incorrectly - ResourceException
    • Unable to publish BIRT reports through ASF

• Platform Project defects impacting the Test Project:
  • Critical/Blocker:
    • Random intermitten errors while launching a test against remote agents: Targeted to I8.
    • command line and runtime workbench unsatisfied link error: Targeted to I8.
    • The edit option is not working properly in probe targets: Resolved.
    • Cannot add method entry fragment data items [regression]: Resolved.
    • Probekit context menu is disabled.Probekit context menu is periodically disabled.: Resolved.
  • Major/Normal:
    • Execution of any testsuite fails using the IAC - InactiveProcessException: Targeted to I8.

• Remaining I8 Bugzillas:
  • Paul:
    • Execution of any plugin junit testsuite fails: Critical issue with running JUnit Plug-in test suites via ASF. Blocking COSMOS' test automation initiative. Blocked by Platform Project defect Execution of any testsuite fails using the IAC - InactiveProcessException.
    • profiling trace generated from running a java app instrumented with api recording probe does not contain information: Reopened from I8 TP1 testing since work-around did not resolve the symptom.
    • Test editor is too wide for users to read: Usability issue. Required this week.
    • Providing the ability to record user's interactions with GEF objects in a GEF editor: Paul will complete the integration, testing, and check-in after the I8 TP2 AGR tests have been completed.
    • Automated Test Execution Harness Tests: Test automation to be completed during TP2.
    • All external APIs should have JUnit tests: Test automation to be completed during TP2.
  • Duwayne:
    • Parallel agent startup errors, InactiveProcessException: Required by a consuming product. Inconsistent test results. Blocked by Platform Project defect Random intermitten errors while launching a test against remote agents.
    • Unable to publish BIRT reports through ASF: Critical issue with generating test reports via ASF. Requires verification using the correct configuration.
    • Test Execution fails when project defined incorrectly - ResourceException: Critical issue with running test suites via ASF.
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.: Test automation to be completed during TP2.
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.: Test automation to be completed during TP2.
  • Jerome:
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.: Test automation to be completed during TP2.
  • Tony:
    • (Linux) AGR test suite does not run.: Test automation to be completed during TP2.

• On today's AG call, we will be discussing the defects that cannot be contained in 4.5.0 but are required by the community or a consuming product, causing us to do a point release (e.g. 4.5.0.1):
  • Paul:
    • Execution of any plugin junit testsuite fails: Critical issue with running JUnit Plug-in test suites via ASF. Blocking COSMOS' test automation initiative. Blocked by Platform Project defect Execution of any testsuite fails using the IAC - InactiveProcessException.
  • Duwayne:
    • Parallel agent startup errors, InactiveProcessException: Required by a consuming product. Blocked by Platform Project defect Random intermitten errors while launching a test against remote agents.

Round-table

Paul

• Last Week:
  • Assisted consuming product triage several symptoms/issues found during their testing.
  • Triaged all 'future' Bugzillas to determine duplicates and relevance.
  • Reviewing patches.
  • Resolved defects:
    • NullPointerException constructing navigator proxies: DefaultTestSuiteProxyNode.addInvocationReferences
    • ClassNotFoundException when running ASF automation services without OSGi/ICU.
    • NullPointerException when switch between resource and logical test navigator
    • Datapools cannot be created/opened without JCA.
    • NullPointerException when generating a Test Execution Tabular Report using ASF.
    • Technical review of Test Project documentation before 4.5 release.
    • Fix spelling/broken links/accessibility errors in JUnit Plugin/Manual Test/BIRT Reports plug-ins.
    • Documentation smoke test failures.
    • (Webdoc (Readme] Datapool variables are not encrypted without the MD5 hash algorithm.]
    • Should be able to use Right Click button on keyboard
    • ExecutionEvent.toString() horribly inefficient
    • Document TPTP Test Project test bucket.
    • Adding to new deployment wizard has duplicate shortcuts
    • Missing title for Test Generation wizard in Resource perspective
    • JScrib test report generators enable the wizard Finish button befrore report name is entered.

• This Week:
  • Action items.
  • Complete I8 TP2.
  • Work on remaining I8 defects.

Duwayne

• Last Week:
  • Out of office: 1 day
  • Assisted consuming product triage several symptoms/issues found during their testing.
  • Reviewing patches.
  • Completed testing.
  • Resolved defect:
    • Cannot remove a test invocation to a removed HTTP request.

• This Week:
  • Action items.
  • Complete I8 TP2.
  • Work on remaining I8 defects.

Jerome

• Last Week:
  • Reviewing patches.
  • Resolved defects:
    • Test assets are allowed to have same, non-unique IDs
    • Unable to delete a test or a project
    • Rename error in java perspective

• This Week:
  • Non-TPTP responsibilities.

Tony

• Last Week:
  • Completed testing.
  • Technical review of the Test Project documentation.
  • Verifying 'future' Bugzillas to determine if the symptoms are reproducible.
  • Testing patches.

• This Week:
  • Action items.
  • Complete I8 TP2.
  • Finish verifying 'future' Bugzillas to determine if the symptoms are reproducible.
  • Work on remaining I8 defects.

Action items

• Duwayne:
  • Reply to your news group posts.
  • Verify/close your resolved defects.
  • Reply to questions in your defects.
  • Add a sizing to your defects.

• Paul:
  • Verify/close your resolved defects.

• Tony:
  • Verify/close your resolved defects.

• Jerome:
  • Verify/close your resolved defects.

Notes

• None.

Reminders

Lead Committers

• Continually review copyright reports and ensure files will missing copyright or incorrect copyright year are corrected.
• Add the Original Estimate (PH) when triaging a new defect.
• Update the Hours Worked (PH) when completing a defect.
• Test their components or review/track testing contributed by other committers/resources.
• Monitor the TPTP news group and answer posts specific to their Test Project components.
• Verify that all API changes (modifications, additions, and removals) do not breech the TPTP API Contract.
• Use the MyFoundation Portal for committer elections.