Difference between revisions of "TPTP-Test-Tools-20080128"

Revision as of 10:32, 28 January 2008

Attendees

• Present:
  • Paul Slauenwhite
  • DuWayne Morris
  • Jerome Bozier
  • Tony Wang

4.4.1

Release Information

• 4.4.1 will ship with the Europa Winter Maintenance Release (target release date: February 29, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• We are currently starting the 4.4.1 test pass (TP) for two weeks (target: February 8).

Testing

• Tony completed the Windows testing as part of his orientation.
• Lead committers have to complete multi-platform and Linux testing for the TP, including:
  • Duwayne:
    • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.Execution.URLRunner.Remote.testsuite
  • Paul:
    • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.smoke.testsuite
    • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.ManualRunner.Windows_XP.testsuite
  • Tony:
    • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/DeploymentsProviderTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/ExecutionHistoryDefaultsProviderTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/facade/DeploymentLaunchConfigurationFacadeTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/facade/ExecutionHistoryLaunchConfigurationFacadeTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/facade/TestLaunchConfigurationFacadeTest.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.Execution.AutoGUIRunner.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.UI.AutoGUIRunner.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.Execution.AutoGUIRunner.Basic.Sun1.4.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.Execution.AutoGUIRunner.Basic.Sun15.testsuite
    • /org.eclipse.hyades.execution.harness.tests/manual/defects/Test.Execution.ExecutionHarness.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.JUnitRunner.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.ManualRunner.Windows_XP.Sun_JRE_142.testsuite
    • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_extensibility.testsuite
    • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_importExport.testsuite
    • /org.eclipse.hyades.test.java.tests/junit/CodeSynchronizationTest.testsuite
    • /org.eclipse.hyades.test.java.tests/junit/JUnitTestSuiteFacadeTest.testsuite
    • /org.eclipse.tptp.test.tools.api.tests/manual/Recorder/Test.UI.APIRecorder.Windows_IA32.Sun_JRE_142.testsuite
    • /org.eclipse.tptp.test.tools.api.tests/manual/Recorder/Test.UI.APIRecorder.Windows_IA32.Sun_JRE_150.testsuite

4.5

Release Information

• 4.5 will ship with the Ganymede Simultaneous Release (target release date: June 30, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• We have delayed the start of I5 testing until TP2 (target: February 11) to focus on the 4.4.1 TP.
• We have extended I5 development to February 6 (February 7/8 for smoke testing in preparation for the I5 TP).

Testing

• Since the 4.4.1 TP overlaps with the 4.5 I5 TP1 (see 4.4.1 Detailed Schedule and 4.5 Detailed Schedule) and there have been sufficient fixes integrated into 4.4.1 to merit a full test pass, we will:
  • Not complete 4.5 I5 TP1 to focus on the 4.4.1 TP.
  • Start the 4.5 I5 TP2 when the 4.4.1 TP is complete.
  • Complete a full test pass for 4.5 I5 TP2.

Test Automation Initiative

• Paul and the Release Engineering Team are working on consolidating our testing process and automation infrastructure:
  • For more information, see the WIKI page.
  • Outstanding defect against the Test Project include:
    • Paul:
      • OutOfMemoryError when generating large TPTP test reports: Completed in i5.
      • Test log not accurate: Targeted to i5.
      • Remove XMLUnit dependency and convert reporting tests to TPTP tests: Targeted to i5.
      • Content missing in Tabular Report: Targeted to i6.
    • Duwayne:
      • ASF Test Execution Service races with AutoBuildJob -- can lead to unexpected test results: Targeted to i6.
      • ANT test results property requires trailing slash to specify location: Targeted to i6.
      • ANT importExistingProjects throws Exception: Targeted to i6.
      • publication task problems: Targeted to i6.
      • Running ASF through Ant requires display export: Targeted to i6.

Enhancements

• Queries.
• Schedule.

• Xin Ying Huang:
  • Encrypted datapool variable support: The implementation is complete. Paul and Joe will be finishing the code review this week

• Duwayne:
  • Test execution should support parallel execution of tests: See the meeting minutes for more details. Duwayne will complete this enhancement in I5/I6 development. The test cases and product documentation will be complete in I5 TPs and I6 development. This past week, Duwayne found non-thread-safe code in the Hyades execution APIs causing exceptions when extracting data from RMI calls. Will consult Joe and Scott for any history on this area of the code.

• Jerome:
  • Move, Copy, Delete, Rename and Import/Export test assets: Jerome is currently working on the engine work. Jerome scheduled a follow-on call/demo for today. For more information, see:
    • Design Review.
    • Design Decisions.
    • Core Use Cases.
    • Detailed Design Document

• Paul:
  • Generic Recorder Wizard enhancement to filter the list of recorders: Paul has closed with the consuming product on removing the duplicate HTTP recording wizard (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=137591 defect 137591) and wizard to import HTTP recordings (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=206078 defect 206078) since both are currently supported by the GRF. Paul will complete the implementation/testing in I5.

Defects

• Queries.
• Schedule.

• Defects requiring a sizing:
  • Duwayne:
    • • publication task problems.
      • Running ASF through Ant requires display export.

• Remaining I5 defects:
  • Paul:
    • Need new icon for testlog files
    • scroll bars not appearing when needed in test suite editor and other Hyades editors when content doesn't all fit in a page
    • (Usability) Switch to Test Perspective after test execution.
    • (Usability) Switch to Test Perspective after creating a test asset.
    • Importing large datapool freezes wizard.
    • Copy / Paste should not be available in the logical test navigator
    • (Usability) Missing/inconsistent titles/descriptions/icons in test report dialogs.
    • (Usability) Test invocation error message cannot be copied
    • Log Viewer : navigation button on event part of test log are not enabled by default
    • TPTP test pass reports show test type ID instead of meaningful name.
    • Remove references to internal APIs.
    • Exported datapool with deleted variables is incorrect.
    • (Usability) Hyperlinks in Test Log view are rendered differently for the Test Execution event type.
    • Report Icon is incorrect for Test Pass and Time Frame Historic
    • Cannot import HTTP recording file.
    • Remove XMLUnit dependency and convert reporting tests to TPTP tests.
    • Duplicate wizards for creating a URL recording/test.
    • Test log not accurate
    • (Test Project) (Web) (Matrix) Update the TPTP Matrix and Bugzilla component owners.
    • Fix project/compilation errors in the TPTP test bucket.
    • Clean-up the TPTP committer lists and project matrix.
  • Duwayne:
    • Vista-specific error dialog when recording using IAC/IE on XP.
    • ASF marshalling return value between service and client adapter does not have support for failures / exceptions
    • Get rid of the Remove option from the Http Requests tab of the Hyades URL Test Suite
    • Text in Recorder Control should be copyable.
    • Repeated launch through Automated Services Framework eventually fails
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
    • RecorderException when HTTP recording with Internet Explorer on Windows XP.
  • Kendric:
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.

Miscellaneous

• (Scalability) Address TPTP EMF model scalability problems: Although withdrawn as a 4.5 candidate enhancement, the outcome/status of the work done in 4.4 on this enhancement was not updated in the WIKI page. Joe will update the WIKI.

• Test project topics for upcoming AG calls include:
  • Enhancement Move, Copy, Delete, Rename and Import/Export test assets: Jerome.
  • Test automation initiative: Alan/Joel
  • EMMA: Paul

Round-table

Paul

• Last Week:
  • Worked on enhancement Generic Recorder Wizard enhancement to filter the list of recorders.
  • Completed defects:
    • Simplified Chinese translation when editing a datapool is incorrect
    • Malformed report window time stamps in the report wizard.
    • Confusing label in Test preferences.
    • Remove the "open editor" button on the Report Wizard first page
    • (Code cleanup) Clean-up translated strings and help contexts from test.ui and test.tools.ui
    • Create Bugzilla components for JUnit plug-in test editor and runner.
    • Formatting messages make translation difficult.
    • (Test Project) Remove duplicate/unused messages
    • (Testing Tool) Invalid characters inserted in source folder textbox when using Browse button of "Generate..." menu item
    • Remove mention of http recorder jars from setconfig
    • Incorrect summary verdict when generating JScrib reports on test executions with no verdicts.
    • OutOfMemoryError when generating large TPTP test reports.
    • Add help icon to Report Wizard.
    • Newly created JUnit tests are sometimes not displayed in the test navigator

• This Week:
  • Enhancement:
    • Generic Recorder Wizard enhancement to filter the list of recorders.
  • Defects:
    • Importing large datapool freezes wizard.
    • Remove references to internal APIs.
    • Exported datapool with deleted variables is incorrect.
    • Cannot import HTTP recording file.
    • Remove XMLUnit dependency and convert reporting tests to TPTP tests.
    • Duplicate wizards for creating a URL recording/test.
    • Test log not accurate
    • Fix project/compilation errors in the TPTP test bucket.

Duwayne

• Last Week:
  • Working on enhancement Test execution should support parallel execution of tests including investigating sequence of events and setting up a test environment for multiple agents.

• This Week:
  • Enhancement:
    • Test execution should support parallel execution of tests.
  • Defect:
    • RecorderException when HTTP recording with Internet Explorer on Windows XP.

Jerome

• Last Week:
• Working on enhancement Move, Copy, Delete, Rename and Import/Export test assets.
  • Completed the Detailed Design Document.

• This Week:
  • Enhancement:
    • Move, Copy, Delete, Rename and Import/Export test assets.
      • Chair follow-on call/demo.

Tony

• Last Week:
  • Completed 4.4.1 test suites on Windows.

• This Week:

Action items

Jerome

• n/a

Notes

News

• Jerome was elected as the Lead Committer for the following components:
  • Test.UI
  • Test.UI.ConfigEditors
  • Test.UI.DatapoolEditor
  • Test.UI.FrameworkEditors
  • Test.UI.TestPerspective

EclipseCon 2008

• Test and Performance Category.
• Test and Performance Presentations

Reminders

Lead Committers

• Continually review copyright reports and ensure files will missing copyright or incorrect copyright year are corrected.
• Add the Original Estimate (PH) when triaging a new defect.
• Update the Hours Worked (PH) when completing a defect.
• Test their components or review/track testing contributed by other committers/resources.
• Monitor the TPTP news group and answer posts specific to their Test Project components.
• Verify that all API changes (modifications, additions, and removals) do not breech the TPTP API Contract.
• Use the MyFoundation Portal for committer elections.