Latest revision as of 06:33, 18 April 2023

p2 Repositories

• Nightly (what Git master targets): http://download.eclipse.org/linuxtools/updates-nightly
• Latest release is always http://download.eclipse.org/linuxtools/update

Release HOWTO

As code is being worked on

1. Write N&N items as new noteworthy stuff is committed.
2. Ensure the project plan is kept up to date with release deliverables and dates. The next release should always be set up after the previous one completes. To add a new release, go to the project plan web page and expand the Committer Tools section. There, you will find Create New Release under the Releases group. Create the release and set the release date according to the release train final release date. The release should be major if APIs are being removed or changed. Otherwise a minor release or bug-fix release is sufficient. A bug-fix only release does not require a review.

ASAP when a release date is known

1. Let project committers and contributors know the cut-off date for accepting patches from non-committers. Reminders on the dates from the table should be send to the mailing list too.

Proposed freeze periods:

3 weeks before the planned release date or RC2 milestone

1. Let the project know that the date has now arrived that new contributions from non-committers cannot be accepted for this release.
2. Email the EMO (emo@eclipse.org) to request a release review and he/she will create a bug
3. If this is the December release
   1. Go over IP Contribution Review to ensure patches have iplog+ where necessary (note: I've gone over everything earlier than bug #315815 and concluded we don't need to do anything -- Andrew Overholt)
   2. Submit IP log with the IP log tool.
4. Prepare the release review documentation and edit the online release plan for the specified release. A lot of information can be copied over from the previous release online documentation. In the past, this was done as a "docuware" document. Those older documents can be found in our website repository's 'doc' sub-directory. The release documentation should have a pointer to our N&N file: https://github.com/eclipse-linuxtools/org.eclipse.linuxtools/blob/master/RELEASE_NOTES.md.

2 weeks before the planned release date or RC2 milestone

1. Create a stable-x branch (ex. stable-5.9) on https://github.com/eclipse-linuxtools/org.eclipse.linuxtools using the branch pull-down
2. On the stable-x branch
   1. modify Jenkinsfile so the branch used is stable-x and the build output is written to updates-nightly-<release> (e.g. 8.5) and updates-docker-nightly-<release> (e.g. 5.5) instead of updates-nightly and updates-docker-nightly (reserved for the bleeding-edge master)
   2. modify the top-level pom.xml so the mirror-repo-name properties have values of update-<version> (e.g. update-8.5.0) and update-docker-<dockerversion> (e.g. update-docker-5.5.0) instead of updates-nightly and updates-docker-nightly (reserved for the bleeding-edge master)
3. On master, update the version number in the pom.xml files (ex. mvn3 org.eclipse.tycho:tycho-versions-plugin:0.17.0:set-version -DnewVersion=${to-be-released-version + 1}-SNAPSHOT; commit and push the resulting changes) Note that you will have to manually change the containers and vagrant directories to change their pom versions
   1. Check if all versions were updated correctly. Usually maven updates more files than we want. We need in this step to update the versions from the pom.xml file from the root directory and from pom.xml files located in 1st level of directories. We don't need to update the version in plugins and features in this step.
4. On master, update the linuxtools-eX.X target file when the new release is started to point to new I-build.
5. Ensure the repos listed in our parent pom.xml contain the dependencies of the correct versions against which we should be building/testing <--- this is important!
6. Commit any work not targetted at the release to master
7. Commit changes specifically targetted towards the release only on the release branch (ex. stable-0.9)
8. Commit changes targetted at both the release and trunk on master and then EGit/User_Guide#Cherry_Picking the commit(s) from master to the branch

10 days before the planned release date or RC2 milestone and this is the December release

1. After IP log has been cleared, save a copy in the website repository (the legal team usually emails a PDF or HTML snapshot to the IP log approval requestor).
2. Email our PMC to request approval for release. A previous request looks like this. This is only required for a major (x.0) or minor ({*.x) release.
   1. Include links to the HTML for the release documentation, IP log and N&N
3. Await PMC approval on tools-pmc mailing list. For the Tools PMC, there is no single approval. What happens is that one or more committers will +1 the request and if no one disputes, then consider these +1s as approvals.
4. Once the PMC has approved the release, email the resulting release review docuware HTML, the IP log snapshot, and a link to the PMC approval email to emo@eclipse.org. If the emo has opened a release tracker bug, you may consider the release approved if the bug has been closed and you have sent the appropriate documentation. In most cases, you will already have supplied the docuware and ip log to the bug.
   1. The EMO may need a week to review the documents, so this step should attempt to finish at least one week to the release day. The emo team usually start the review process at Wednesdays. So it is a good idea to finish this step before the Wednesday that it at least one week from the release date. If this won't be possible, ask them about deadlines.

A few days before the release or RC2 milestone

1. All component leads should now be satisfied that the branch is stable and all unit tests should be passing
2. Remove the "-SNAPSHOT" from pom.xml versions (find . -name pom.xml | xargs sed -i '/-parent/,+3 s/-SNAPSHOT//1')
   • We used to use mvn3 org.eclipse.tycho:tycho-versions-plugin:set-version -DnewVersion=${to-be-released-version} but the maven command removes -SNAPSHOT from feature and plugin versions and their corresponding .qualifier references in feature.xml and MANIFEST.MF. Regardless, you should look at the build to make sure all features/plug-ins have date stamps.
3. Ensure all p2.inf files found in features, point to the Linux Tools stable update site. To do a mass change, one could do: find . -name p2.inf | xargs sed -i -e 's/updates-nightly/update/g'; find . -name p2.inf | xargs sed -i -e 's/updates-docker-nightly/update-docker/g'. The master branch will point to updates-nightly and updates-docker-nightly by default. A branch already used for a stable release will likely already be pointed to the correct update site and no action needs to be taken. Look in an existing feature such as autotools/org.eclipse.linuxtools.autotools-feature for confirmation.
   • In the case of a release which will be contributed to a simultaneous release, this step needs to be done in advance of the RC4 contribution build.
4. With the version set properly in the repository's pom.xml files, push a build in Jenkins. Note that if the release is also targeted for a simultaneous release, then the last build submitted for RC2 must be the release build.
5. Verify the p2 repository
   • A handy way to list the contents of a p2 repository is: eclipse -consolelog -nosplash -application org.eclipse.equinox.p2.director -repository <url> -list
   • Ensure all qualifiers are the same and are what you expect
   • Using the p2 UI, ensure the IUs are categorized properly
6. Perform a quick smoke test to ensure the build is acceptable. Zero code changes should have happened since the SHA-1 that was used for the previous build.
7. Ensure Equinox/p2/p2.mirrorsURL has been automatically set in artifacts.xml (in artifacts.jar) and that the p2.index file exists.
8. If things are acceptable with the signed build, tag the Git repo:
   • use a tag of the format vMajor.Minor.Micro (ex. v0.7.1)
   • use the -a flag on the tag
   • specify --tags on the git push command to ensure that the tag is visible to others
   • do not tag until the IP Log is completed and the repository has been tested
   • if you tag too early, you can tag again using -a -f to force the re-tag, however, if you re-tag, you need to send a note out to indicate this has happened
   • once complete, an announcement note should be sent out to tell the list that the tag has been created and for what commit SHA-1 the tag is for.
9. Lock the Hudson job that was used for the release build to prevent automatic deletion (ex. build #263) and add a description (ex. "0.7.1 release")
10. Save the archive of the entire build somewhere before it gets over-written (it will not be stored across the next build)
    • ex. wget --no-check-certificate https://ci.eclipse.org/linuxtools/job/linuxtools/job/stable-8.6/4/artifact/*zip*/archive.zip
11. Get source tarballs:
    1. Go to https://github.com/eclipse-linuxtools/org.eclipse.linuxtools/releases/tag/vX.Y.Z
    2. Download the .tar.gz file for the matching tag to the Linux Tools downloads area (need to have special access rights)
    3. rename the tarball to linuxtools-${version}.tar.gz
    4. move the source tarball to http://download.eclipse.org/technology/linuxtools/${version}-sources/, copying and modifying index.php from a previous release
    5. chmod 755 the source tarball

Release day

1. Put an archive of the entire build into the downloads directory (~/downloads/technology/linuxtools) (done via Jenkins job)
   1. Unzip the archive.zip saved after last successful build
   2. cd to sub-directory containing actualy repository and re-zip contents to linuxtools-RELEASE.zip in the downloads directory
   3. Run md5sum linuxtools-RELEASE-incubation.zip >linuxtools-RELEASE.zip.md5.
   4. move the previous linuxtools-RELEASE.zip and md5 to /home/data/httpd/archive.eclipse.org/linuxtools
   5. chgrp technology.linux-distros linuxtools-RELEASE.zip*
   6. chmod 755 linuxtools-RELEASE.zip*
2. Update the link on downloads.php to point to the correct listing of source tarballs for this release
3. Update the "Unit test results" link on downloads.php to point to the Hudson job that was used
4. Add a news item to the main wiki page
5. Ensure download links -- including the zip of the repository -- are correct
6. Re-verify the p2 repository
   1. A handy way to list the contents of a p2 repository is: eclipse -consolelog -nosplash -application org.eclipse.equinox.p2.director -repository <url> -list
   2. Ensure all qualifiers are the same and are what you expect
   3. Using the p2 UI, ensure the IUs are categorized properly
7. Copy our p2 repository to a versioned copy to archive it (ex. cp -rp updates-nightly-8.5.0 update-8.5.0)
8. Update any relevant Eclipse Marketplace pages
   1. https://marketplace.eclipse.org/content/eclipse-docker-tooling
   2. https://marketplace.eclipse.org/content/eclipse-vagrant-tooling
9. Announce release on mailing list and blog(s)
10. Mark release as complete in list of releases in portal
11. Add next version to list of releases in portal

Day after the release

1. Relax :)
2. Decide whether or not older releases should be moved to archive.eclipse.org (take note of p2.mirrorsURL modification)
3. Switch the Jenkins job back to use the master or stable-*** branch
4. Start on the next release

Simultaneous Release Inclusion

As of Helios, Linux Tools is a part of the annual Eclipse simultaneous release. The simultaneous release aggregator takes content from the p2 repos that are listed in our b3aggrcon files. Builds that we would like to promote to the simultaneous release must:

• be signed
• not necessarily be in category.xml
• exist in the p2 repository listed in the linuxtools.b3aggrcon file with the exact same feature versions/qualifiers

Note also that categories for the main simultaneous release are different from our p2 repository categories (which are set in our category.xml).

The builds must remain in the p2 repo until we change the feature versions/qualifiers in the b3aggrcon file. This will prevent future aggregation runs from failing. More information can be found here:

• Simrel/Contributing_to_Simrel_Aggregation_Build <-- This one contains the git location for org.eclipse.simrel.build where the linuxtools.b3aggrcon file lives in the master branch and will need to be updated to new qualifiers [and versions] when we want to include a new release in Simrel

The cross-project-issues-dev mailing list must be monitored by project leads and release engineers. This mailing list can help with any problems with the simultaneous release as well as with EPP packages. David Williams leads the simultaneous release creation and Markus Knauer coordinates the EPP packages including "ours" (Indigo).

Note that during the "quiet week" between RC2 and the final release, our release bits must be put into their final place but not be made "visible". Read the Final Daze document for guidelines and pointers to FAQs on how to make things invisible, etc.

Adding a new component

Our build process is pom.xml-driven but we have a mirrored feature structure for use by p2 repos. In order for a new component to be built, it needs to fit into the hierarchy somewhere. When adding a new top-level sub-project feature (sub-features of existing features should get added to their containing feature), follow these steps:

Code-level checklist

1. Create your feature and containing plugins
   1. Name the feature(s) org.eclipse.linuxtools.mycoolstuff and put it in Git as org.eclipse.linuxtools.mycoolstuff-feature (replacing "mycoolstuff", obviously)
   2. Name the plugin(s) org.eclipse.linuxtools.mycoolstuff.core, org.eclipse.linuxtools.mycoolstuff.ui, etc.
   3. Do not put _ characters in the bundle ID (this breaks the Maven signing plugin we're using)
   4. Ensure your packages are all in the org.eclipse.linuxtools namespace (and in the .mycoolstuff.core, .mycoolstuff.ui packages where appropriate)
   5. Ensure your strings are externalized
   6. Ensure your feature and plugin provider fields are set to "Eclipse Linux Tools" (no quotes)
2. Either copy over existing pom.xml files and manually edit them or generate using Tycho and manually edit
3. Copy over existing p2.inf file from any other feature and add p2.inf to your build.properties binary files
4. Create your JUnit test plugins
   1. Name your test plugin the same as your functional plugin but with a ".tests" tacked onto the end
   2. Ensure your test bundle's pom.xml looks like an existing test bundle's pom.xml
5. Enable API Tools on non-example/test/feature plugins

Git-level checklist

1. If this is a new sub-project, create a directory to contain it, like "gcov" or "gprof"
2. Check all of your new stuff into Git master

Build-related checklist

1. Add any new top-level feature to the top-level pom.xml in your Git clone
2. If your sub-project has dependencies outside the existing ones (BIRT, EMF, DTP, CDT, GEF), notify the mailing list and project leads
   1. Hopefully this will have been caught in the CQ (legal review)
3. Ensure your BREEs are correct in your plugin MANIFEST.MF files
4. Ensure the version on your feature ends with ".qualifier" (without the quotation marks)
5. Ensure the versions in your MANIFEST.MF and feature.xml files match those in your pom.xml files
6. Add new features to our p2 repo's category.xml file
7. Ensure your pom.xml files have the same source plugin and feature bits as the others
8. Run a full local build from the top-level of your git clone with mvn -fae clean install to ensure the build still works

Building locally

• ensure you have a recent Maven 3 release on your path
• clone our Git repository (perhaps read our Git instructions)
• cd into your clone and run mvn -fae clean install
• You can add the parameter "-Dmaven.test.skip=true" for debug purposes, but remember that no contribution that breaks the unit tests will be accepted.
• Some components have dependencies between then, for example Valgrind plug-ins. In that case, always build the whole Valgrind suite instead of one particular plug-in.
• follow this guide for debugging tests being run by Tycho

Adding a new test plugin to those that are run during the automated build

• Create test plugin(s) (ex. org.eclipse.linuxtools.mycoolfeature.ui.tests)
• Copy an existing test plugin's pom.xml (this is used when the automated build is run)
• Add your test plugin to the parent pom.xml
• Check your plugin(s) into Git
• Verify that your tests are built/run with a local build (see instructions on this page)

The next time a build happens, your test plugin(s) will be built and run. If you need a build pushed sooner than the next 6 hour mark when our scheduled builds happen, speak with the project leads via linuxtools-dev@eclipse.org.