Difference between revisions of "TPTP-Test-Tools-20080128"

Latest revision as of 07:36, 11 February 2008

Attendees

• Present:
  • Paul Slauenwhite
  • DuWayne Morris
  • Jerome Bozier
  • Tony Wang
  • Harm Sluiman

4.4.1

Release Information

• 4.4.1 will ship with the Europa Winter Maintenance Release (target release date: February 29, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• We are currently starting the 4.4.1 test pass (TP) for two weeks (target: February 8).

Testing

• Tony completed a majority of the Windows testing as part of his orientation.
• The candidate driver will be announced by the Release Engineering Team later today.
• The outstanding testing includes:
  • Duwayne:
    • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.Execution.URLRunner.Remote.testsuite
  • Paul:
    • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.smoke.testsuite
    • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.ManualRunner.Windows_XP.testsuite
  • Tony:
    • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/DeploymentsProviderTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/ExecutionHistoryDefaultsProviderTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/facade/DeploymentLaunchConfigurationFacadeTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/facade/ExecutionHistoryLaunchConfigurationFacadeTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/facade/TestLaunchConfigurationFacadeTest.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.Execution.AutoGUIRunner.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.UI.AutoGUIRunner.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.Execution.AutoGUIRunner.Basic.Sun1.4.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.Execution.AutoGUIRunner.Basic.Sun15.testsuite
    • /org.eclipse.hyades.execution.harness.tests/manual/defects/Test.Execution.ExecutionHarness.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.JUnitRunner.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.ManualRunner.Windows_XP.Sun_JRE_142.testsuite
    • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_extensibility.testsuite
    • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_importExport.testsuite
    • /org.eclipse.hyades.test.java.tests/junit/CodeSynchronizationTest.testsuite
    • /org.eclipse.hyades.test.java.tests/junit/JUnitTestSuiteFacadeTest.testsuite
    • /org.eclipse.tptp.test.tools.api.tests/manual/Recorder/Test.UI.APIRecorder.Windows_IA32.Sun_JRE_142.testsuite
    • /org.eclipse.tptp.test.tools.api.tests/manual/Recorder/Test.UI.APIRecorder.Windows_IA32.Sun_JRE_150.testsuite

4.5

Release Information

• 4.5 will ship with the Ganymede Simultaneous Release (target release date: June 30, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• We have delayed the start of I5 testing until TP2 (target: February 11) to focus on the 4.4.1 TP.
• We have extended I5 development to February 6 (February 7/8 for smoke testing in preparation for the I5 TP) by attaching patches to the Bugzilla and requesting Test Project approval to be included in I5.

Testing

• Since the 4.4.1 TP overlaps with the 4.5 I5 TP1 (see 4.4.1 Detailed Schedule and 4.5 Detailed Schedule) and there have been sufficient fixes integrated into 4.4.1 to merit a full test pass, we will:
  • Not complete 4.5 I5 TP1 to focus on the 4.4.1 TP.
  • Start the 4.5 I5 TP2 when the 4.4.1 TP is complete.
  • Complete a full test pass for 4.5 I5 TP2.

Test Automation Initiative

• Paul and the Release Engineering Team are working on consolidating our testing process and automation infrastructure:
  • For more information, see the WIKI page.
  • Outstanding defect against the Test Project include:
    • Paul:
      • OutOfMemoryError when generating large TPTP test reports: Completed in i5.
      • Content missing in Tabular Report: Targeted to i5.
      • Test log not accurate: Targeted to i5.
      • Remove XMLUnit dependency and convert reporting tests to TPTP tests: Targeted to i5.
    • Duwayne:
      • ASF Test Execution Service races with AutoBuildJob -- can lead to unexpected test results: Targeted to i6.
      • ANT test results property requires trailing slash to specify location: Targeted to i6.
      • ANT importExistingProjects throws Exception: Targeted to i6.
      • publication task problems: Targeted to i6.
      • Running ASF through Ant requires display export: Targeted to i6.

Enhancements

• Queries.
• Schedule.

• Xin Ying Huang:
  • Encrypted datapool variable support: The implementation is complete. Paul and Joe will be finishing the code review this week.

• Duwayne:
  • Test execution should support parallel execution of tests: See the meeting minutes for more details. Duwayne will complete this enhancement in I5/I6 development. The test cases and product documentation will be complete in I5 TPs and I6 development. This past week, Duwayne found some serious design issues (or perhaps partial implementation) that need to be fixed to handle the RMI return data properly. The timing of calling thread activities can cause an error where the caller does not get the right return data. Duwayne will work on the RMI return data multi-caller design details to properly join the RMI return data to the right caller at the appropriate time, try to avoid any added serialization of activities perhaps avoiding the current notifyAll() (waking up everybody when there is only data available for one caller thread), and testing. Duwayne will schedule a call for next week to discuss the progress and outstanding items.

• Jerome:
  • Move, Copy, Delete, Rename and Import/Export test assets: Jerome is currently working on the engine work. Jerome scheduled a follow-on call/demo for today. For more information, see:
    • Design Review.
    • Design Decisions.
    • Core Use Cases.
    • Detailed Design Document

• Paul:
  • Generic Recorder Wizard enhancement to filter the list of recorders: Paul has closed with the consuming product on removing the duplicate HTTP recording wizard (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=137591 defect 137591) and wizard to import HTTP recordings (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=206078 defect 206078) since both are currently supported by the GRF. Paul will complete the implementation/testing in I5.

Defects

• Queries.
• Schedule.

• Defects requiring a sizing:
  • Duwayne:
    • • publication task problems.
      • Running ASF through Ant requires display export.

• Defects required by consuming products in I5:
  • Duwayne:
    • Repeated launch through Automated Services Framework eventually fails

• Remaining I5 defects:
  • Paul:
    • Need new icon for testlog files
    • scroll bars not appearing when needed in test suite editor and other Hyades editors when content doesn't all fit in a page
    • (Usability) Switch to Test Perspective after test execution.
    • (Usability) Switch to Test Perspective after creating a test asset.
    • Importing large datapool freezes wizard.
    • Copy / Paste should not be available in the logical test navigator
    • (Usability) Missing/inconsistent titles/descriptions/icons in test report dialogs.
    • (Usability) Test invocation error message cannot be copied
    • Log Viewer : navigation button on event part of test log are not enabled by default
    • TPTP test pass reports show test type ID instead of meaningful name.
    • Remove references to internal APIs.
    • Exported datapool with deleted variables is incorrect.
    • (Usability) Hyperlinks in Test Log view are rendered differently for the Test Execution event type.
    • Report Icon is incorrect for Test Pass and Time Frame Historic
    • Cannot import HTTP recording file.
    • Remove XMLUnit dependency and convert reporting tests to TPTP tests.
    • Duplicate wizards for creating a URL recording/test.
    • Test log not accurate
    • (Test Project) (Web) (Matrix) Update the TPTP Matrix and Bugzilla component owners.
    • Fix project/compilation errors in the TPTP test bucket.
    • Clean-up the TPTP committer lists and project matrix.
  • Duwayne:
    • Vista-specific error dialog when recording using IAC/IE on XP.
    • ASF marshalling return value between service and client adapter does not have support for failures / exceptions
    • Get rid of the Remove option from the Http Requests tab of the Hyades URL Test Suite
    • Text in Recorder Control should be copyable.
    • Repeated launch through Automated Services Framework eventually fails
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
    • RecorderException when HTTP recording with Internet Explorer on Windows XP.
  • Kendric:
    • (Linux) AGR test suite does not run.
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.

Miscellaneous

• (Scalability) Address TPTP EMF model scalability problems: Although withdrawn as a 4.5 candidate enhancement, the outcome/status of the work done in 4.4 on this enhancement was not updated in the WIKI page. Joe will update the WIKI.

• Test project topics for upcoming AG calls include:
  • Enhancement Move, Copy, Delete, Rename and Import/Export test assets: Jerome.
  • Test automation initiative: Alan/Joel
  • EMMA: Paul

Round-table

Paul

• Last Week:
  • Worked on enhancement Generic Recorder Wizard enhancement to filter the list of recorders.
  • Completed defects:
    • Simplified Chinese translation when editing a datapool is incorrect
    • Malformed report window time stamps in the report wizard.
    • Confusing label in Test preferences.
    • Remove the "open editor" button on the Report Wizard first page
    • (Code cleanup) Clean-up translated strings and help contexts from test.ui and test.tools.ui
    • Create Bugzilla components for JUnit plug-in test editor and runner.
    • Formatting messages make translation difficult.
    • (Test Project) Remove duplicate/unused messages
    • (Testing Tool) Invalid characters inserted in source folder textbox when using Browse button of "Generate..." menu item
    • Remove mention of http recorder jars from setconfig
    • Incorrect summary verdict when generating JScrib reports on test executions with no verdicts.
    • OutOfMemoryError when generating large TPTP test reports.
    • Add help icon to Report Wizard.
    • Newly created JUnit tests are sometimes not displayed in the test navigator

• This Week:
  • 4.4.1 TP.
  • Enhancement:
    • Generic Recorder Wizard enhancement to filter the list of recorders.
  • Defects:
    • Importing large datapool freezes wizard.
    • Remove references to internal APIs.
    • Exported datapool with deleted variables is incorrect.
    • Cannot import HTTP recording file.
    • Remove XMLUnit dependency and convert reporting tests to TPTP tests.
    • Duplicate wizards for creating a URL recording/test.
    • Test log not accurate
    • Fix project/compilation errors in the TPTP test bucket.

Duwayne

• Last Week:
  • Working on enhancement Test execution should support parallel execution of tests including investigating sequence of events and setting up a test environment for multiple agents.

• This Week:
  • 4.4.1 TP.
  • Enhancement:
    • Test execution should support parallel execution of tests.
  • Defect:
    • RecorderException when HTTP recording with Internet Explorer on Windows XP.

Jerome

• Last Week:
• Working on enhancement Move, Copy, Delete, Rename and Import/Export test assets.
  • Completed the Detailed Design Document.

• This Week:
  • Enhancement:
    • Move, Copy, Delete, Rename and Import/Export test assets.
      • Chair follow-on call/demo.

Tony

• Last Week:
  • Completed the assigned 4.4.1 test suites on Windows.

• This Week:
  • Complete the outstanding 4.4.1 test suites on Windows.

Action items

Duwayne

• Schedule a call for next week to discuss the progress and outstanding items for enhancement Test execution should support parallel execution of tests.

Notes

News

• Jerome was elected as the Lead Committer for the following components:
  • Test.UI
  • Test.UI.ConfigEditors
  • Test.UI.DatapoolEditor
  • Test.UI.FrameworkEditors
  • Test.UI.TestPerspective

EclipseCon 2008

• Test and Performance Category.
• Test and Performance Presentations

Reminders

Lead Committers

• Continually review copyright reports and ensure files will missing copyright or incorrect copyright year are corrected.
• Add the Original Estimate (PH) when triaging a new defect.
• Update the Hours Worked (PH) when completing a defect.
• Test their components or review/track testing contributed by other committers/resources.
• Monitor the TPTP news group and answer posts specific to their Test Project components.
• Verify that all API changes (modifications, additions, and removals) do not breech the TPTP API Contract.
• Use the MyFoundation Portal for committer elections.