Difference between revisions of "TPTP-Test-Tools-20080512"

Revision as of 07:26, 12 May 2008

Attendees

• Present:
  • Paul Slauenwhite
  • DuWayne Morris
  • Tony Wang
  • Jerome Bozier

4.5

Release Information

• 4.5 will ship with the Ganymede Simultaneous Release (target release date: June 30, 2008).

• Detailed Schedule.

• Defect queries.

• Download.

• Test reports.

Where are we at?

• This week is the first week of I8 TP1 (target: May 23).
  • All defects require patch reviews, Stop Ship Template, and Project/PMC approval.

Testing

• We want to complete the I8 TP1 by Friday, May 16.
• The candidate driver will be announced by the Release Engineering Team later today.

4.5 I8 TP1

• A note has been sent to the originators of all resolved defects reminding them to verify/close these defects before Ganymede releases.

• Please verify/close your resolved defects during the TP1:
  • Paul:
    • • (Trace Project) Create Bugzilla component for trace reports.
      • TPTP test results are not reported correctly.
      • Incorrect project names in .project files.
      • Fix broken externalized strings for the Monitoring Project.
      • CTRL-S does not work in the XML Source tab of the *.rptdesgin editor.
      • Support capturing additional draw2d related events when recording user's interactions with a GEF editor.
      • No data is recorded when recording after a hung/cancelled test execution.
      • URL Recorder operational messages require cleanup.
      • BIRT Trace Report does not appear in the Trace Report wizard.
      • Instrument the Data Collection Engine to log native Common Base Events.
      • Incorrect dependencies in TPTP features when installing from the Callisto Discovery Site.
      • Warnings in .log file.
      • 4.4.0.3 copyright build report is incorrect.
      • Provide reference for CDT in Requirements section of the download page.
      • Update project sets for TPTP 4.5.
      • Refactor TPTP project sets for component reorganization.
      • (Vista) File paths are too long when unzipping all-in-one.
      • Add a 'Bundle-ActivationPolicy' header to all manifest.mf files.
      • Duplicate record error when import a CSV into an existing datapool.
      • sun.io.MalformedInputException when deleting a test asset in the Test Navigator.
      • (Functionality, Usability) Testing resources cannot be renamed, deleted and/or moved without breaking associations.
      • Update Platform section of TPTP Project Matrix.
      • Missing link.
      • Exceptions when opening the Platform documentation.
      • (Platform Project) Create Bugzilla component for statistical reports.
      • TPTP 4.5 release notes are empty.
      • TPTP test results are not reported correctly.
      • Leverage/integrate PIN for line level code coverage of TPTP's native code.
      • Update copyright dates.
      • 4.4 API change report is not accurate.
      • Reinstate features and 'empty' plug-ins for as-is components.
      • (Web) Test execution periodically hangs.
      • (Web) (Vista) Test execution terminates abnormally.
      • Workbench location wizard switches values for -arch and -ws.
      • Update copyright dates and translatability errors.
      • (Web) (Linux) Cannot not delete test assets in the Test Navigator.
      • Exception in the deployment editor when a referenced artifact is deleted.
      • (ASF) FileNotFoundException when interrogating test execution results.
      • Resolving a boolean value from a datapool always returns false.
      • Generating a URL test class in the root Java package causes a compilation error.
      • Datapool cells are not created for CSV ending with a comma.
      • Recorder Control view is not opened or has focus when a recording is started
      • Test Tools about.properties files contain incorrect descriptions.
      • Importing large datapool freezes wizard.
      • Move JUnit Plug-in test tools documentation to a separate plug-in.
      • Clicking duplicate topic headings jumps to the first instance.
      • Create TVT/enablement test cases for new Test Project UI.
      • Vista-specific error dialog when recording using IAC/IE on XP.
      • NullPointerException when deleting artifact location pairs.
      • Time out error message during test execution is not resolving.
      • (Usability) Missing/inconsistent titles/descriptions/icons in test report dialogs.
      • Deprecate JScrib and JScrib report generators.
      • Test Tools documentation I5/I6 smoke test failures.
      • System default browser is not used as the default browser for URL recording.
      • Extra .execution appended to the results property.
      • Remove references to internal APIs.
      • Exported datapool is incorrect.
      • Cannot import HTTP recording file.
      • (Usability) Logical and physical names for test assets is confusing.
      • Failures in /org.eclipse.hyades.test.ui.datapool.tests/junit/datapool/Test.UI.EncryptedDatapoolEditor_running.testsuite.
      • ClassNotFoundException when running ASF automation services without OSGi/ICU.
      • Duplicate wizards for creating a URL recording/test.
      • UI allows to launch non-profilable tests in profile mode
      • Leverage/integrate EMMA for line level code coverage of TPTP's Java code.
      • Fix broken externalized strings for the Trace Project.
      • (Platform Project) Remove duplicate/unused messages
      • (Trace Project) Remove duplicate/unused messages
      • Eventformatter does not correctly serialize the any properties.
      • Exception when de-serializing nested Common Base Event XML fragments/documents.
      • Remove legacy IBM/Rational comment/string references.
      • Remove Hyades references from logging examples.
      • API Recorder's Generate Test Suite dialog does not select a default selection.
      • (Usability) Switch to Test Perspective after creating a test asset.
      • Test.Execution.JUnitRunner.testsuite and Test.Execution.JUnitRunner.Bug186196.testsuite are failing.
      • Cannot install BIRT without WTP with Eclipse M6.
      • Connection refused when lauching tests on Linux.
      • IAC does not configure plug-in's pluginconfig.xml at startup.
  • Tony:
    • • Connection refused on local host during test execution.
      • The 'Encrypt' option doesn't work with keyboard selection
      • "Paste" doesn't not complete normally
      • The challenge is not removed from the datapool when the last encrypted variable is removed
      • Datapool can not be created with more than one variable
      • The invocation for some deleted testcases can't be removed in behaviour tab
      • IAC can't record a Java application
  • Duwayne:
    • • Test Navigator doesn't always show new assets
      • (regression) tptp:test service fails when using the suite parameter
      • Broken strings in ASF command shell adapter
      • Consumming Product: ASF Throws Modal Dialog Box.
      • SSL Recording not working, regression
      • Test Execution Fails
      • (Vista) HTTP Recorder requires work-around on Windows Vista for Internet Explorer
      • All In One TPTP Package Does not have Test Perspective
      • All-in-One build for Linux File Permissions Issue
      • Regression, test execution on Vista crashes the AC
      • IAC will not start on Linux IA32
      • Extra java.exe left running after each HTTP Recording
      • Documentation for HTTP Recording with IE 7
      • (Web Doc) IPV6 not supported for Internet Explorer
      • (Web Doc) IPv6 Limitation on Firefox
      • Regression, URL Test Does Not Get Generated
      • Import fails if project name contains special characters
      • Foreign Language RAC Install Error
      • Regression in RAC, Hyades URL Test fails to execute properly
      • HTTP Recorder does not work with IAC in TPTP 4.3.1 and 4.4
      • non-externalized strings in org.eclipse.hyades.execution
      • IAC broken for URL Tests on 4.3
  • Jerome:
    • • persisted proxy fail to load
      • log viewer : change adress preference for defect is not applied to already entered defect
      • lack of error message when remote execution fail

• Please use the TPTP all-in-one package and install dependent/As-Is/Technology Preview components in the <Eclipse installation directory>/eclipse/dropins directory.

• TP1 testing assignments:
  • Manual:
    • Tony:
      • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_importExport.testsuite
      • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.EncryptedDatapool_editing.testsuite
      • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.DatapoolEditor_editing.testsuite
      • /org.eclipse.hyades.test.ui.datapool.tests/manual/datapool/Test.UI.Datapool_Running.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_linux.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_windows.testsuite
      • /org.eclipse.hyades.test.ui.logViewer.tests/manual/Test.UI.FrameworkEditors_logViewer_extensibility.testsuite
      • /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_bugzilla_166025_namecheck.testsuite
      • /org.eclipse.hyades.test.ui.navigator.tests/manual/Test.UI.TestNavigator_deployment_check.testsuite
    • Paul:
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.Datapools.technical_review.Windows_IA32.testsuite
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.Deployment.technical_review.Windows_IA32.testsuite
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.General.technical_review.Windows_IA32.testsuite
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.JUnit.technical_review.Windows_IA32.testsuite
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.JUnit_Plugin.technical_review.Windows_IA32.testsuite
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.Manual.technical_review.Windows_IA32.testsuite
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.Test_Log_Viewer.technical_review.Windows_IA32.testsuite
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.smoke.testsuite
      • /org.eclipse.hyades.execution.harness.tests/manual/defects/Test.Execution.ExecutionHarness.testsuite
      • /org.eclipse.hyades.test.java.tests/manual/regressionTests/Test.Execution.JUnitPluginRunner.testsuite
      • /org.eclipse.tptp.test.tools.api.tests/manual/Recorder/Test.UI.APIRecorder.Windows_IA32.testsuite
    • Duwayne:
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.URL.technical_review.Windows_IA32.testsuite
      • /org.eclipse.tptp.test.doc.user/manual/Doc/Test.Doc.ASF.technical_review.Windows_IA32.testsuite
      • /org.eclipse.hyades.test.core.services.tests/manual/Test.Execution.Services.testsuite
      • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.Windows.testsuite
      • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.UI.URLTest.Linux.testsuite
      • /org.eclipse.hyades.test.URLTest.tests/manual/basic_tests/Test.Execution.URLRunner.Remote.testsuite
  • JUnit:
    • Tony:
      • /org.eclipse.hyades.test.ui.logViewer.tests/JUnit/Test.UI.FrameworkEditors_logViewer_UtilitiesTest.testsuite
      • /org.eclipse.hyades.test.ui.datapool.tests/junit/datapool/Test.UI.DatapoolEditor_running.testsuite
      • /org.eclipse.hyades.test.ui.datapool.tests/junit/datapool/Test.UI.EncryptedDatapoolEditor_running.testsuite
    • Duwayne:
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.IBM16_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.Sun16_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.IBM142_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.IBM15_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.Sun142_TestServices.testsuite
      • /org.eclipse.tptp.test.testservices.tests/junit/Test.Sun15_TestServices.testsuite
  • JUnit Plug-in:
    • Paul:
      • /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.CodeSynchronizationTest.testsuite
      • /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.JUnitTestGeneration.testsuite
      • /org.eclipse.hyades.test.java.tests/junit/Test.JUnit.JUnitTestSuiteFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/facade/TestLaunchConfigurationFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/facade/DeploymentLaunchConfigurationFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/facade/ExecutionHistoryLaunchConfigurationFacadeTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/DeploymentsProviderTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/ExecutionHistoryDefaultsProviderTest.testsuite
      • /org.eclipse.hyades.test.core.tests/junit/launch/extensions/DatapoolHandlerTest.testsuite
    • Tony:
      • /org.eclipse.hyades.test.ui.datapool.tests/junit_plugin/datapool/Test.UI.Datapool_Running.testsuite
      • /org.eclipse.hyades.test.ui.datapool.tests/junit_plugin/datapool/Test.UI.EncryptedDatapool_Running.testsuite
  • AGR:
    • Paul:
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.IBM16_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.Sun16_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.IBM142_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.IBM15_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.Sun142_JUnitRunner.testsuite
      • /org.eclipse.hyades.test.java.tests/autoui/Test.Execution.Sun15_JUnitRunner.testsuite
    • Tony:
      • /org.eclipse.hyades.test.ui.navigator.tests/gui/Test.UI.TestNavigator_Logical_auto.testsuite
      • /org.eclipse.hyades.test.ui.navigator.tests/gui/Test.UI.TestNavigator_Resource_auto.testsuite
      • /org.eclipse.hyades.test.ui.datapool.tests/gui/Test.UI.EncryptedDatapoolEditor.testsuite

News Group

• The following news group posts require a reply by Friday, May 16:
  • Duwayne:
    • Re: Using wait command in AGR script (March 11, 2008 2:07 PM)
    • Re: HTTP Recording stops immediately after start-up (FireFox) or never records data (IE) (March 25, 2008 1:15 PM)
    • tptp execuation task issue (April 2, 2008 3:12 PM)
    • Multi-Language tests with AGR (April 7, 2008 3:11 PM)
    • Testing an application with Custom component (April 8, 2008 11:03 AM)
    • Re: NoClassDefFoundError: org.eclipse.hyades.test.common...? (April 10, 2008 5:33 PM)
    • Re: Multiple instances for Verification Class (April 28, 2008 3:11 PM)
    • Re: Reusable Thread Error in standard mode (April 29, 2008 6:12 PM)

Test Automation Initiative

• Paul and the Release Engineering Team are working on consolidating our testing process and automation infrastructure:
  • For more information, see the WIKI page.
  • To see the BVT results for a specific build, open the Build Reports page for a specific build and select BVT Results.
  • Outstanding defect against the Platform Project include:
    • Execution of any testsuite fails using the IAC - InactiveProcessException
  • Outstanding defect against the Test Project include:
    • Paul:
      • JUnit Plug-in runner incorrectly modifies deployed plugin.xml files.
      • AGR Tests require absolute path to Equinox JAR
      • Clear remote workspace launch does not work for remote test execution
      • AGR Tests cannot be run with start ACServer
      • Remove XMLUnit dependency and convert reporting tests to TPTP tests
      • Content missing in Tabular Report
      • Connection property does not deploy all dependencies
      • Connection property does not allow JUnit Plug-in Tests to be executed
    • Duwayne:
      • ASF marshalling return value between service and client adapter does not have support for failures / exceptions
      • Confusing exceptions when running automated services.
      • ASF Test Execution Service races with AutoBuildJob -- can lead to unexpected test results
      • ANT test results property requires trailing slash to specify location
      • Exceptions when running ASF tasks.
      • Only the name of the execution result is used from the results property.

Enhancements

• Queries.
• Schedule.

• Alexander Nyßen:
  • Providing the ability to record user's interactions with GEF objects in a GEF editor: Alexander (TPTP user and contributor) has contributed a patch and documentation for basic GEF/Draw2D support in the AGR. AG/PMC approved and Contribution Questionnaire (CQ) approved. Paul did not complete the integration, testing, and check-in during I8. Since the AGR is an As-Is component, Paul will complete this integration after we complete I8 TP1.

Defects

• Queries.

• Schedule.

• Defects require a sizing:
  • Jerome:
    • Test Navigator link to editor doesn't work when multiple editors open on file mapping to same proxy node.

• Defects to be deferred to Future:
  • See AG Call Minutes.

• Platform Project impacting the Test Project:
  • Execution of any testsuite fails using the IAC - InactiveProcessException: Bing cannot reproduce. Working with originator to determine if a configuration issue.

• Defects required in I8:
  • Paul:
    • Provide ISV documentation for enhancement 166025.: Deferred to Future
    • TPTP EMF models needs to update: Deferred to Future
    • Context sensitive help requires refactoring and new content.: Deferred to Future
    • (Usability) HTTP tests do not automatically generate JUnit Java code like JUnit tests.: Deferred to Future
    • Test editor is too wide for users to read: Will be resolved during I8 TP1
    • Technical review of Test Project documentation before 4.5 release.: Will be resolved during I8 TP1
    • Fix spelling/broken links/accessibility errors in JUnit Plugin/Manual Test/BIRT Reports plug-ins.: Will be resolved during I8 TP1
    • Documentation smoke test failures.: Will be resolved during I8 TP1
    • Providing the ability to record user's interactions with GEF objects in a GEF editor: Will be resolved during I8 TP1
    • Review all plug-in dependencies.: Will be resolved during I8 TP1
    • Clean-up the TPTP committer lists and project matrix.: Will be resolved during I8 TP1
    • Regression, URL Test Does Not Get Generated: Resolved
    • Starting Recording forces change to Test Perspective: Resolved
    • Need new icon for testlog files: Resolved
    • Creating resources with # in name is not prevented but causes problems loading the model: Resolved
    • URL test type does not support encrypted datapools.: Resolved
    • Junit plug-in test type does not support encrypted datapools.: Resolved
    • Resolving a boolean value from a datapool always returns false.: Resolved
    • Generating a URL test class in the root Java package causes a compilation error.: Resolved
    • 'asked for password' dialog still pop up when running the test after encrypt--then decrypt all variables in datapool (no encrypted data when running): Resolved
    • The challenge is not removed from the datapool when the last encrypted variable is removed: Resolved
    • Test Tools about.properties files contain incorrect descriptions.: Resolved
    • Deprecate JScrib and JScrib report generators.: Resolved
    • testgen wizard does not preselect options: Resolved
    • profiling trace generated from running a java app instrumented with api recording probe does not contain information: Resolved
    • ClassCastException with DPLVariable in DatapoolTable: Resolved
    • Valid IPv6 Addresses produce 'Invalid IP Address' error in a Test Location: Resolved
    • lack of error message when remote execution fail: Resolved
    • (Usability) Switch to Test Perspective after test execution.: Resolved
    • Extra .execution appended to the results property.: Resolved
    • Datapool cells are not created for CSV ending with a comma.: Resolved
    • NoClassDefFound junit TestCase: Resolved
  • Duwayne:
    • SSL Recording not working, regression
    • Random intermitten errors while launching a test against remote agents
    • Parallel agent startup errors, InactiveProcessException
    • Exception when canceling a recording (before "recording "actually starts)
    • Cannot remove a test invocation to a removed HTTP request.: Patch is attached for one symptom (single test invocation), but solution for symptom with multiple test invocations is outstanding.
  • Jerome:
    • Test Navigator link to editor doesn't work when multiple editors open on file mapping to same proxy node.
    • Unable to create proxy for testsuite
  • Tony:
    • Accessibility-Need shortcuts for all Test Navigator pulldowns: Problems with shortcuts for on Test Navigator context menus.
  • Kit:
    • Incorrect german translation of the word "Pass"

• Defects deferred to I8 TP1/TP2:
  • Tony:
    • [Linux AGR test suite does not run.]
  • Duwayne:
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
  • Jerome:
    • (Test Automation) Automate manual test suites using the Automated GUI Recorder.
  • Paul:
    • All external APIs should have JUnit tests
    • Automated Test Execution Harness Tests

Round-table

Paul

• Last Week:
  • Resolved defects:
    • • Deprecate JScrib and JScrib report generators.
      • Regression, URL Test Does Not Get Generated
      • profiling trace generated from running a java app instrumented with api recording probe does not contain information
      • ClassCastException with DPLVariable in DatapoolTable
      • Valid IPv6 Addresses produce 'Invalid IP Address' error in a Test Location
      • 'asked for password' dialog still pop up when running the test after encrypt--then decrypt all variables in datapool (no encrypted data when running)
      • Resolving a boolean value from a datapool always returns false.
      • Generating a URL test class in the root Java package causes a compilation error.
      • Starting Recording forces change to Test Perspective
      • Creating resources with # in name is not prevented but causes problems loading the model
      • Need new icon for testlog files
      • lack of error message when remote execution fail
      • Junit plug-in test type does not support encrypted datapools.
      • URL test type does not support encrypted datapools.
      • Test Tools about.properties files contain incorrect descriptions.
      • The challenge is not removed from the datapool when the last encrypted variable is removed
      • testgen wizard does not preselect options
      • (Usability) Switch to Test Perspective after test execution.
      • Extra .execution appended to the results property.
      • Datapool cells are not created for CSV ending with a comma.
      • NoClassDefFound junit TestCase
      • Duplicate record error when import a CSV into an existing datapool.
      • Problem with testing JSF application.

• This Week:
  • 4.5 I8 TP1.

Duwayne

• Last Week:
  • Resolved defect:
    • • SSL Recording not working, regression

• This Week:
  • 4.5 I8 TP1.

Jerome

• Last Week:
  • Resolved defect:
    • • Refactoring error on attempt to Delete multiple Results from TN view.

• This Week:

Tony

• Last Week:

• This Week:
  • 4.5 I8 TP1.

Action items

• None.

Notes

• None.

Reminders

Lead Committers

• Continually review copyright reports and ensure files will missing copyright or incorrect copyright year are corrected.
• Add the Original Estimate (PH) when triaging a new defect.
• Update the Hours Worked (PH) when completing a defect.
• Test their components or review/track testing contributed by other committers/resources.
• Monitor the TPTP news group and answer posts specific to their Test Project components.
• Verify that all API changes (modifications, additions, and removals) do not breech the TPTP API Contract.
• Use the MyFoundation Portal for committer elections.