Difference between revisions of "TPTP-Test-Tools-20080204"

Revision as of 10:53, 4 February 2008

Attendees

• Present:
  • Paul Slauenwhite
  • DuWayne Morris
  • Jerome Bozier
  • Tony Wang

4.4.1

Release Information

• 4.4.1 will ship with the Europa Winter Maintenance Release (target release date: February 29, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• We are currently starting the last week of the 4.4.1 test pass (TP) (target: February 8).

Testing

• Candidate driver: TPTP-4.4.1-200801280906

• Blocker:
  • (Linux) Cannot connect to the Agent Controller: Paul will verify the fix today in the TPTP-4.4.1-200802010100 driver.

• Tony completed a most of the 4.4.1 testing as part of his orientation. He did an exceptional job completing the testing on-time and without errors. A job well done Tony!

• The outstanding testing includes:
  • Duwayne:
    • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.Execution.URLRunner.Remote.testsuite
  • Paul:
    • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.smoke.testsuite
    • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.ManualRunner.Windows_XP.testsuite
  • Tony:
    • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/DeploymentsProviderTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/ExecutionHistoryDefaultsProviderTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/facade/DeploymentLaunchConfigurationFacadeTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/facade/ExecutionHistoryLaunchConfigurationFacadeTest.testsuite
    • /org.eclipse.hyades.test.core.tests/junit/launch/facade/TestLaunchConfigurationFacadeTest.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.Execution.AutoGUIRunner.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.UI.AutoGUIRunner.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.Execution.AutoGUIRunner.Basic.Sun1.4.testsuite
    • /org.eclipse.tptp.test.auto.gui.tests/manual/Test.Execution.AutoGUIRunner.Basic.Sun15.testsuite
    • /org.eclipse.hyades.execution.harness.tests/manual/defects/Test.Execution.ExecutionHarness.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.JUnitRunner.testsuite
    • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.ManualRunner.Windows_XP.Sun_JRE_142.testsuite
    • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_extensibility.testsuite
    • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_importExport.testsuite
    • /org.eclipse.hyades.test.java.tests/junit/CodeSynchronizationTest.testsuite
    • /org.eclipse.hyades.test.java.tests/junit/JUnitTestSuiteFacadeTest.testsuite
    • /org.eclipse.tptp.test.tools.api.tests/manual/Recorder/Test.UI.APIRecorder.Windows_IA32.Sun_JRE_142.testsuite
    • /org.eclipse.tptp.test.tools.api.tests/manual/Recorder/Test.UI.APIRecorder.Windows_IA32.Sun_JRE_150.testsuite

4.5

Release Information

• 4.5 will ship with the Ganymede Simultaneous Release (target release date: June 30, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• We are starting a 1.5 week I5 TP1 (target: February 13) since the 4.4.1 TP is nearly complete.
• I5 development is closed and requires Project/PMC approval.

Testing

• We want to complete the I5 TP1 by Friday, February 8.
• The candidate driver will be announced by the Release Engineering Team later today.

4.5 I5 TP1

• TP1 testing assignments:
  • Manual:
    • Tony:
      • org.eclipse.hyades.test.ui.datapool.tests\manual\datapool\Test.UI.DatapoolEditor_editing.testsuite
      • org.eclipse.hyades.test.ui.datapool.tests\manual\datapool\Test.UI.DatapoolEditor_importExport.testsuite
      • org.eclipse.hyades.test.ui.logViewer.tests\manual\Test.UI.FrameworkEditors_logViewer_linux.testsuite
      • org.eclipse.hyades.test.ui.logViewer.tests\manual\Test.UI.FrameworkEditors_logViewer_windows.testsuite
      • org.eclipse.hyades.test.ui.logViewer.tests\manual\Test.UI.FrameworkEditors_logViewer_extensibility.testsuite
      • org.eclipse.hyades.execution.harness.tests\manual\defects\Test.Execution.ExecutionHarness.testsuite
    • Paul:
      • org.eclipse.tptp.test.doc.user\manual\Doc\Test.Doc.smoke.testsuite
      • org.eclipse.tptp.test.tools.api.tests\manual\Recorder\Test.UI.APIRecorder.Windows_IA32.testsuite
      • org.eclipse.tptp.platform.report.tests\manual\Reporting\Plaform.UI.Charting_ChartBoundsSVG.testsuite
      • org.eclipse.tptp.platform.report.tests\manual\Reporting\Plaform.UI.Charting_ChartInternationlizationSVG.testsuite
      • org.eclipse.tptp.platform.report.tests\manual\Reporting\Plaform.UI.Charting_ChartStaticImage.testsuite
      • org.eclipse.tptp.platform.report.tests\manual\Reporting\Plaform.UI.Charting_ChartSVG.testsuite
      • org.eclipse.tptp.platform.report.tests\manual\Reporting\Plaform.UI.Charting_ChartSWT.testsuite
      • org.eclipse.hyades.test.java.tests\manual\regressionTests\Test.Execution.JUnitPluginRunner.testsuite
    • Duwayne:
      • org.eclipse.hyades.test.URLTest.tests\manual\basic_tests\Test.UI.URLTest.Windows.testsuite
      • org.eclipse.hyades.test.URLTest.tests\manual\basic_tests\Test.UI.URLTest.Linux.testsuite
      • org.eclipse.hyades.test.URLTest.tests\manual\basic_tests\Test.Execution.URLRunner.Remote.testsuite
  • JUnit:
    • Paul:
      • org.eclipse.tptp.test.testservices.tests\junit\Test.IBM142_TestServices.testsuite
      • org.eclipse.tptp.test.testservices.tests\junit\Test.IBM15_TestServices.testsuite
      • org.eclipse.tptp.test.testservices.tests\junit\Test.Sun142_TestServices.testsuite
      • org.eclipse.tptp.test.testservices.tests\junit\Test.Sun15_TestServices.testsuite
    • Tony:
      • org.eclipse.hyades.test.ui.logViewer.tests\JUnit\Test.UI.FrameworkEditors_logViewer_UtilitiesTest.testsuite
  • JUnit Plug-in:
    • Paul:
      • org.eclipse.hyades.test.java.tests\junit\Test.JUnit.CodeSynchronizationTest.testsuite
      • org.eclipse.hyades.test.java.tests\junit\Test.JUnit.JUnitTestGeneration.testsuite
      • org.eclipse.hyades.test.java.tests\junit\Test.JUnit.JUnitTestSuiteFacadeTest.testsuite
      • org.eclipse.hyades.test.core.tests\junit\launch\facade\TestLaunchConfigurationFacadeTest.testsuite
      • org.eclipse.hyades.test.core.tests\junit\launch\facade\DeploymentLaunchConfigurationFacadeTest.testsuite
      • org.eclipse.hyades.test.core.tests\junit\launch\facade\ExecutionHistoryLaunchConfigurationFacadeTest.testsuite
      • org.eclipse.hyades.test.core.tests\junit\launch\extensions\DeploymentsProviderTest.testsuite
      • org.eclipse.hyades.test.core.tests\junit\launch\extensions\ExecutionHistoryDefaultsProviderTest.testsuite
  • AGR:
    • Paul:
      • org.eclipse.hyades.test.java.tests\autoui\Test.Execution.IBM142_JUnitRunner.testsuite
      • org.eclipse.hyades.test.java.tests\autoui\Test.Execution.IBM15_JUnitRunner.testsuite
      • org.eclipse.hyades.test.java.tests\autoui\Test.Execution.Sun142_JUnitRunner.testsuite
      • org.eclipse.hyades.test.java.tests\autoui\Test.Execution.Sun15_JUnitRunner.testsuite
    • Tony:
      • org.eclipse.hyades.test.ui.logViewer.tests\gui\Test.UI.FrameworkEditors_logViewer_linux.testsuite
      • org.eclipse.hyades.test.ui.logViewer.tests\gui\Test.UI.FrameworkEditors_logViewer_windows.testsuite
      • org.eclipse.hyades.test.ui.navigator.tests\gui\TestNavigator_Logical_auto.testsuite
      • org.eclipse.hyades.test.ui.navigator.tests\gui\TestNavigator_Resource_auto.testsuite

Test Automation Initiative

• Paul and the Release Engineering Team are working on consolidating our testing process and automation infrastructure:
  • On Friday, Paul and Joel were able to run JUnit/JUnit Plug-in BVTs on the Reference Platform. The next step is for Joel to integrate these BVTs with the nightly build process and post the Test Reports.
  • For more information, see the WIKI page.
  • Outstanding defect against the Test Project include:
    • Paul:
      • Test log not accurate: Completed in i5.
      • publication task problems: Completed in i5.
      • Remove XMLUnit dependency and convert reporting tests to TPTP tests: Targeted to i5.
      • Content missing in Tabular Report: Targeted to i6.
    • Duwayne:
      • ASF Test Execution Service races with AutoBuildJob -- can lead to unexpected test results: Targeted to i6.
      • ANT test results property requires trailing slash to specify location: Targeted to i6.
      • ANT importExistingProjects throws Exception: Targeted to i6.
      • Running ASF through Ant requires display export: Targeted to i6.

Enhancements

• Queries.
• Schedule.

• Xin Ying Huang:
  • Encrypted datapool variable support: The implementation is complete. Paul and Joe will be finishing the code review this week.

• Duwayne:
  • Test execution should support parallel execution of tests: See the meeting minutes for more details. Duwayne will complete this enhancement in I5/I6 development. The test cases and product documentation will be complete in I5 TPs and I6 development. This past week, Duwayne found some serious design issues (or perhaps partial implementation) that need to be fixed to handle the RMI return data properly. The timing of calling thread activities can cause an error where the caller does not get the right return data. Duwayne will work on the RMI return data multi-caller design details to properly join the RMI return data to the right caller at the appropriate time, try to avoid any added serialization of activities perhaps avoiding the current notifyAll() (waking up everybody when there is only data available for one caller thread), and testing. Duwayne will schedule a call for next week to discuss the progress and outstanding items.

• Jerome:
  • Move, Copy, Delete, Rename and Import/Export test assets: Jerome is currently working on the engine work. Jerome scheduled a follow-on call/demo for today. For more information, see:
    • Design Review.
    • Design Decisions.
    • Core Use Cases.
    • Detailed Design Document

• Paul:
  • Generic Recorder Wizard enhancement to filter the list of recorders: Paul has closed with the consuming product on removing the duplicate HTTP recording wizard (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=137591 defect 137591) and wizard to import HTTP recordings (see [https://bugs.eclipse.org/bugs/show_bug.cgi?id=206078 defect 206078) since both are currently supported by the GRF. Paul will complete the implementation/testing in I6.

Defects

• Queries.
• Schedule.

• Defects requiring a sizing:
  • Duwayne:
    • • publication task problems.
      • Running ASF through Ant requires display export.
  • Jerome:
    • • Copying lots of test results to a newly created folder hangs the system
      • Wrong behaviour when using Working Sets in Test Navigator

• Defects required by consuming products in I5:
  • Duwayne:
    • Repeated launch through Automated Services Framework eventually fails

• Remaining I5 defects:
  • Paul:
    • Remove XMLUnit dependency and convert reporting tests to TPTP tests.
  • Duwayne:
    • Repeated launch through Automated Services Framework eventually fails

• The following I5 defects have been deferred to I6:
  • • ASF marshalling return value between service and client adapter does not have support for failures / exceptions
    • RecorderException when HTTP recording with Internet Explorer on Windows XP.
    • Vista-specific error dialog when recording using IAC/IE on XP.
    • Text in Recorder Control should be copyable.
    • Get rid of the Remove option from the Http Requests tab of the Hyades URL Test Suite
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
    • Duplicate wizards for creating a URL recording/test.
    • Copy / Paste should not be available in the logical test navigator
    • (Usability) Hyperlinks in Test Log view are rendered differently for the Test Execution event type.
    • Remove references to internal APIs.
    • Exported datapool with deleted variables is incorrect.
    • Importing large datapool freezes wizard.
    • scroll bars not appearing when needed in test suite editor and other Hyades editors when content doesn't all fit in a page
    • Log Viewer : navigation button on event part of test log are not enabled by default
    • TPTP test pass reports show test type ID instead of meaningful name.
    • Need new icon for testlog files
    • Cannot import HTTP recording file.
    • (Usability) Switch to Test Perspective after test execution.
    • (Usability) Test invocation error message cannot be copied
    • (Usability) Switch to Test Perspective after creating a test asset.
    • Report Icon is incorrect for Test Pass and Time Frame Historic
    • (Usability) Missing/inconsistent titles/descriptions/icons in test report dialogs.

Miscellaneous

• (Scalability) Address TPTP EMF model scalability problems: Although withdrawn as a 4.5 candidate enhancement, the outcome/status of the work done in 4.4 on this enhancement was not updated in the WIKI page. Joe will update the WIKI.

• Test project topics for upcoming AG calls include:
  • Enhancement Move, Copy, Delete, Rename and Import/Export test assets: Jerome.
  • Test automation initiative: Alan/Joel
  • EMMA: Paul

Round-table

Paul

• Last Week:
  • Worked on enhancement Generic Recorder Wizard enhancement to filter the list of recorders.
  • Completed defects:
    • (Test Project) (Web) (Matrix) Update the TPTP Matrix and Bugzilla component owners.
    • Ant impoertExistingProjects task does not build projects
    • publication task problems
    • Fix project/compilation errors in the TPTP test bucket.
    • Platform.Communication.Installation.Uninstallation testsuite failed to run
    • Test log not accurate

• This Week:
  • 4.4.1 TP.
  • 4.5 I5 TP.
  • Defect:
    • Remove XMLUnit dependency and convert reporting tests to TPTP tests.
  • Finish reviewing:
    • Encrypted datapool variable support
    • Move, Copy, Delete, Rename and Import/Export test assets

Duwayne

• Last Week:
  • Working on enhancement Test execution should support parallel execution of tests including investigating sequence of events and setting up a test environment for multiple agents.

• This Week:
  • 4.4.1 TP.
  • 4.5 I5 TP.
  • Enhancement:
    • Test execution should support parallel execution of tests.
  • Defect:
    • Repeated launch through Automated Services Framework eventually fails

Jerome

• Last Week:
• Working on enhancement Move, Copy, Delete, Rename and Import/Export test assets.
  • Completed the Detailed Design Document.

• This Week:
  • Enhancement:
    • Move, Copy, Delete, Rename and Import/Export test assets.
      • Chair follow-on call/demo.

Tony

• Last Week:
  • Completed the assigned 4.4.1 test suites on Windows.

• This Week:
  • 4.5 I5 TP.

Action items

Duwayne

• Schedule a call for next week to discuss the progress and outstanding items for enhancement Test execution should support parallel execution of tests.

Notes

News

• The PMC approved Jerome as the Lead Committer for the following components:
  • Test.UI
  • Test.UI.ConfigEditors
  • Test.UI.DatapoolEditor
  • Test.UI.FrameworkEditors
  • Test.UI.TestPerspective

• Paul created a Test Tools Project Code Snippets page (linked from the Miscellaneous page in the TPTP WIKI) for code snippets that demonstrate specific programmatic tasks commonly requested by Test Tools users on the news group or mailing list.

EclipseCon 2008

• Test and Performance Category.
• Test and Performance Presentations

Reminders

Lead Committers

• Continually review copyright reports and ensure files will missing copyright or incorrect copyright year are corrected.
• Add the Original Estimate (PH) when triaging a new defect.
• Update the Hours Worked (PH) when completing a defect.
• Test their components or review/track testing contributed by other committers/resources.
• Monitor the TPTP news group and answer posts specific to their Test Project components.
• Verify that all API changes (modifications, additions, and removals) do not breech the TPTP API Contract.
• Use the MyFoundation Portal for committer elections.