Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "Common Build Infrastructure/Publishing"
(New page: OK, so you've built your SDK or runtime zip, your examples, your docs, your tests, and a p2 repo / update site. Great. How do you get those bits out to the world? Well, for the simple cas...) |
(→Example: VE Nightly and Integration Builds: update script to set JAVA_HOME and ANT_HOME) |
||
(17 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
+ | {{:DocumentationGuidelines/DraftHeader}} | ||
+ | |||
OK, so you've built your SDK or runtime zip, your examples, your docs, your tests, and a p2 repo / update site. Great. How do you get those bits out to the world? | OK, so you've built your SDK or runtime zip, your examples, your docs, your tests, and a p2 repo / update site. Great. How do you get those bits out to the world? | ||
− | Well, for the simple case of "built on build.eclipse.org, published over NFS to download.eclipse.org", you can use a simple Ant script, run from your crontab on build.eclipse.org. | + | Well, for the simple case of "built on build.eclipse.org, published over NFS to download.eclipse.org", you can use a simple Ant script, run from your [http://en.wikipedia.org/wiki/Cron#crontab_syntax crontab] on build.eclipse.org. |
+ | |||
+ | # Copy sample promote.xml and promote.properties from org.eclipse.ve.releng or other Athana project. | ||
+ | # Update the promote.properties to fit your needs. | ||
+ | # Add entry in your crontab that will invoke the promote.xml Ant script. | ||
=== Example: VE Nightly and Integration Builds === | === Example: VE Nightly and Integration Builds === | ||
+ | [[Image:Hudson-Config-Parameterized-Build.png|right|thumb]] | ||
Say you want to run Nightlies every day, and a single Integration build once a week. First, set up two jobs in Hudson: | Say you want to run Nightlies every day, and a single Integration build once a week. First, set up two jobs in Hudson: | ||
Line 10: | Line 17: | ||
# https://build.eclipse.org/hudson/view/Athena%20CBI/job/cbi-ve-1.4-integration-Ganymede/ | # https://build.eclipse.org/hudson/view/Athena%20CBI/job/cbi-ve-1.4-integration-Ganymede/ | ||
− | (To save time, create the Nightly job, then copy it to produce the Integration build, passing in slightly different parameters like BUILDTYPE=N or BUILDTYPE=I.) | + | (To save time, create the Nightly job, then copy it to produce the Integration build, passing in slightly different parameters like <code>BUILDTYPE = N</code> or <code>BUILDTYPE = I</code>.) |
− | Next, create crontab entries which match the frequency of your N and I builds. In this example, the latest Nightly VE build will be published every day at 15h40 EST. The latest Integration will be published every Tuesday at 16h00 EST. Note that the promote.xml is identical, but the properties file differs. | + | Next, ssh to build.eclipse.org using your committer id and type <code>crontab -e</code> to create crontab entries which match the frequency of your N and I builds. In this example, the latest Nightly VE build will be published every day at 15h40 EST. The latest Integration will be published every Tuesday at 16h00 EST. Note that the promote.xml is identical, but the properties file differs. |
# daily N builds | # daily N builds | ||
− | 40 15 * * * / | + | 40 15 * * * JAVA_HOME=/shared/common/ibm-java-x86_64-60 ANT_HOME=/shared/common/apache-ant-1.7.1/ ant -f |
/opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-nightly-Ganymede/workspace/build/org.eclipse.ve.releng/promote.xml | /opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-nightly-Ganymede/workspace/build/org.eclipse.ve.releng/promote.xml | ||
-Dpromote.properties=/opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-nightly-Ganymede/workspace/build/org.eclipse.ve.releng/promote-N.properties | -Dpromote.properties=/opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-nightly-Ganymede/workspace/build/org.eclipse.ve.releng/promote-N.properties | ||
Line 21: | Line 28: | ||
# weekly I build | # weekly I build | ||
− | 00 16 * * 2 / | + | 00 16 * * 2 JAVA_HOME=/shared/common/ibm-java-x86_64-60 ANT_HOME=/shared/common/apache-ant-1.7.1/ ant -f |
/opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-integration-Ganymede/workspace/build/org.eclipse.ve.releng/promote.xml | /opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-integration-Ganymede/workspace/build/org.eclipse.ve.releng/promote.xml | ||
-Dpromote.properties=/opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-integration-Ganymede/workspace/build/org.eclipse.ve.releng/promote-I.properties | -Dpromote.properties=/opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-integration-Ganymede/workspace/build/org.eclipse.ve.releng/promote-I.properties | ||
2>/dev/null 1> ~/promo_logs/cbi-ve-1.4-integration-Ganymede-promo.txt | 2>/dev/null 1> ~/promo_logs/cbi-ve-1.4-integration-Ganymede-promo.txt | ||
− | In the N build properties file, we use "add" mode to incrementally publish N builds as they happen. In the I properties file, we use "replace" mode to scrub any accumulated builds and publish only the latest I build. So, once a week, we automatically purge the old garbage and get a fresh build. | + | In the [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ve/org.eclipse.ve.releng/promote-N.properties?root=Tools_Project&view=markup N build properties file], we use "add" mode to incrementally publish N builds as they happen. In the [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ve/org.eclipse.ve.releng/promote-I.properties?root=Tools_Project&view=markup I properties file], we use "replace" mode to scrub any accumulated builds and publish only the latest I build. So, once a week, we automatically purge the old garbage and get a fresh build. |
In addition to publishing your zips, the promote.xml script also unpacks your Update zip into the path you specify, so you will have a daily/weekly up to date update site. | In addition to publishing your zips, the promote.xml script also unpacks your Update zip into the path you specify, so you will have a daily/weekly up to date update site. | ||
+ | |||
+ | ===Specifying Source Directory=== | ||
+ | |||
+ | Your promote.*.properties file needs to define the parent folder in which to find a build folder. For example: | ||
+ | |||
+ | sourceZipsDir=/tmp/build | ||
+ | |||
+ | if building locally, or | ||
+ | |||
+ | sourceZipsDir=/opt/users/hudsonbuild/.hudson/jobs/'''HUDSON_JOB_NAME'''/lastSuccessfulBuild/archive/build | ||
+ | |||
+ | if building on build.eclipse.org, where '''HUDSON_JOB_NAME''' = cbi-ve-1.5-Helios, cbi-tmf-xtext-0.7, etc. | ||
+ | |||
+ | However, since lastSuccessfulBuild or lastStableBuild symlinks may not work with the latest version of Hudson, a better way is needed ({{bug|302170}}). For a <code>sourceZipsDir</code> such as: | ||
+ | |||
+ | sourceZipsDir=/opt/users/hudsonbuild/.hudson/jobs/'''HUDSON_JOB_NAME'''/ | ||
+ | |||
+ | Athena will attempt to find your build dir based on three well-known patterns, for all <code>buildTypes</code> specified (eg., N, I, M, S and/or R): | ||
+ | |||
+ | /opt/users/hudsonbuild/.hudson/jobs/'''HUDSON_JOB_NAME'''/N201001031122 | ||
+ | /opt/users/hudsonbuild/.hudson/jobs/'''HUDSON_JOB_NAME'''/2010-01-03_11-22-33/archive/build/N201001031122 | ||
+ | /opt/users/hudsonbuild/.hudson/jobs/'''HUDSON_JOB_NAME'''/workspace/build/N201001031122 | ||
+ | |||
+ | If more than one match is found, the newest directory by name (most recent build) will be selected. | ||
+ | |||
+ | As a result, if your lastest build contains test errors/failures, you will need to ensure that you do not accidentally promote that build (unless you want to do so). To avoid this, simply set a more explicit <code>sourceZipsDir</code> such as: | ||
+ | |||
+ | sourceZipsDir=/opt/users/hudsonbuild/.hudson/jobs/'''HUDSON_JOB_NAME'''/2010-01-03_11-22-33/archive/build | ||
+ | |||
+ | If you always want to publish your latest build, use: | ||
+ | |||
+ | sourceZipsDir=/opt/users/hudsonbuild/.hudson/jobs/'''HUDSON_JOB_NAME'''/workspace/build | ||
===References=== | ===References=== | ||
+ | * {{bug|273302}}, [https://bugs.eclipse.org/bugs/show_bug.cgi?id=270849#c37 bug 270849], {{bug|284593}}, {{bug|302170}} | ||
+ | * [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/promote-wrapper.xml?root=Technology_Project per-project promote.xml wrapper] | ||
* [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/promote.xml?root=Technology_Project common promote.xml] | * [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.dash/athena/org.eclipse.dash.commonbuilder/org.eclipse.dash.common.releng/promote.xml?root=Technology_Project common promote.xml] | ||
* [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ve/org.eclipse.ve.releng/?root=Tools_Project org.eclipse.ve.releng] | * [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ve/org.eclipse.ve.releng/?root=Tools_Project org.eclipse.ve.releng] | ||
+ | :* [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ve/org.eclipse.ve.releng/promote.xml?root=Tools_Project&view=markup per-project promote.xml wrapper] | ||
:* [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ve/org.eclipse.ve.releng/promote-I.properties?root=Tools_Project&view=markup promote-I.properties] | :* [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ve/org.eclipse.ve.releng/promote-I.properties?root=Tools_Project&view=markup promote-I.properties] | ||
:* [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ve/org.eclipse.ve.releng/promote-N.properties?root=Tools_Project&view=markup promote-N.properties] | :* [http://dev.eclipse.org/viewcvs/index.cgi/org.eclipse.ve/org.eclipse.ve.releng/promote-N.properties?root=Tools_Project&view=markup promote-N.properties] | ||
+ | |||
+ | [[Category:Athena Common Build]] | ||
+ | [[Category:Draft_Documentation]] |
Latest revision as of 02:17, 14 December 2010
OK, so you've built your SDK or runtime zip, your examples, your docs, your tests, and a p2 repo / update site. Great. How do you get those bits out to the world?
Well, for the simple case of "built on build.eclipse.org, published over NFS to download.eclipse.org", you can use a simple Ant script, run from your crontab on build.eclipse.org.
- Copy sample promote.xml and promote.properties from org.eclipse.ve.releng or other Athana project.
- Update the promote.properties to fit your needs.
- Add entry in your crontab that will invoke the promote.xml Ant script.
Example: VE Nightly and Integration Builds
Say you want to run Nightlies every day, and a single Integration build once a week. First, set up two jobs in Hudson:
- https://build.eclipse.org/hudson/view/Athena%20CBI/job/cbi-ve-1.4-nightly-Ganymede/
- https://build.eclipse.org/hudson/view/Athena%20CBI/job/cbi-ve-1.4-integration-Ganymede/
(To save time, create the Nightly job, then copy it to produce the Integration build, passing in slightly different parameters like BUILDTYPE = N
or BUILDTYPE = I
.)
Next, ssh to build.eclipse.org using your committer id and type crontab -e
to create crontab entries which match the frequency of your N and I builds. In this example, the latest Nightly VE build will be published every day at 15h40 EST. The latest Integration will be published every Tuesday at 16h00 EST. Note that the promote.xml is identical, but the properties file differs.
# daily N builds 40 15 * * * JAVA_HOME=/shared/common/ibm-java-x86_64-60 ANT_HOME=/shared/common/apache-ant-1.7.1/ ant -f /opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-nightly-Ganymede/workspace/build/org.eclipse.ve.releng/promote.xml -Dpromote.properties=/opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-nightly-Ganymede/workspace/build/org.eclipse.ve.releng/promote-N.properties 2>/dev/null 1> ~/promo_logs/cbi-ve-1.4-nightly-Ganymede-promo.txt
# weekly I build 00 16 * * 2 JAVA_HOME=/shared/common/ibm-java-x86_64-60 ANT_HOME=/shared/common/apache-ant-1.7.1/ ant -f /opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-integration-Ganymede/workspace/build/org.eclipse.ve.releng/promote.xml -Dpromote.properties=/opt/users/hudsonbuild/.hudson/jobs/cbi-ve-1.4-integration-Ganymede/workspace/build/org.eclipse.ve.releng/promote-I.properties 2>/dev/null 1> ~/promo_logs/cbi-ve-1.4-integration-Ganymede-promo.txt
In the N build properties file, we use "add" mode to incrementally publish N builds as they happen. In the I properties file, we use "replace" mode to scrub any accumulated builds and publish only the latest I build. So, once a week, we automatically purge the old garbage and get a fresh build.
In addition to publishing your zips, the promote.xml script also unpacks your Update zip into the path you specify, so you will have a daily/weekly up to date update site.
Specifying Source Directory
Your promote.*.properties file needs to define the parent folder in which to find a build folder. For example:
sourceZipsDir=/tmp/build
if building locally, or
sourceZipsDir=/opt/users/hudsonbuild/.hudson/jobs/HUDSON_JOB_NAME/lastSuccessfulBuild/archive/build
if building on build.eclipse.org, where HUDSON_JOB_NAME = cbi-ve-1.5-Helios, cbi-tmf-xtext-0.7, etc.
However, since lastSuccessfulBuild or lastStableBuild symlinks may not work with the latest version of Hudson, a better way is needed (bug 302170). For a sourceZipsDir
such as:
sourceZipsDir=/opt/users/hudsonbuild/.hudson/jobs/HUDSON_JOB_NAME/
Athena will attempt to find your build dir based on three well-known patterns, for all buildTypes
specified (eg., N, I, M, S and/or R):
/opt/users/hudsonbuild/.hudson/jobs/HUDSON_JOB_NAME/N201001031122 /opt/users/hudsonbuild/.hudson/jobs/HUDSON_JOB_NAME/2010-01-03_11-22-33/archive/build/N201001031122 /opt/users/hudsonbuild/.hudson/jobs/HUDSON_JOB_NAME/workspace/build/N201001031122
If more than one match is found, the newest directory by name (most recent build) will be selected.
As a result, if your lastest build contains test errors/failures, you will need to ensure that you do not accidentally promote that build (unless you want to do so). To avoid this, simply set a more explicit sourceZipsDir
such as:
sourceZipsDir=/opt/users/hudsonbuild/.hudson/jobs/HUDSON_JOB_NAME/2010-01-03_11-22-33/archive/build
If you always want to publish your latest build, use:
sourceZipsDir=/opt/users/hudsonbuild/.hudson/jobs/HUDSON_JOB_NAME/workspace/build