|
|
(7 intermediate revisions by the same user not shown) |
Line 1: |
Line 1: |
− | = Graphiti Build Process =
| + | The information on this page was completely outdated and thus removed. |
− | | + | |
− | This page describes the Graphiti build setup. Thanks to the Teneo and CDO guys for providing a template for setting up a first version of our build and an excellent description of it (the [http://wiki.eclipse.org/Teneo/Teneo_Build_Setup Teneo build description] was also used as a template for this page). Graphiti uses [http://www.eclipse.org/buckminster/ Buckminster] and the [http://wiki.eclipse.org/Hudson Eclipse Hudson] for its continuous build. The job [https://hudson.eclipse.org/hudson/job/gmp-graphiti-nightly/ gmp-graphiti-nightly] is used to build Graphiti's head version. To understand the content of this page it is necessary to understand the main Buckminster concepts. These are described in detail in the [http://www.eclipse.org/downloads/download.php?file=/tools/buckminster/doc/BuckyBook.pdf Bucky Book]. Some Buckminster concepts can seem fairly abstract initially, the 'Understanding Buckminster' section below may help here.
| + | |
− | | + | |
− | <h4>Local Execution</h4><br><p>It is possible to do a local Graphiti build. To download Graphiti<br>and do a build (building, testing, generating p2 site) you need to do<br>the following steps:</p><br><ul><br> <li>Import the Graphiti releng project from the Eclipse CVS (from<br> dev.eclipse.org:/cvsroot/modeling/org.eclipse.gmp/org.eclipse.gmp.graphiti/releng/org.eclipse.graphiti.releng)<br> into your Eclipse IDE (Buckminster must be installed).</li><br> <li>Start the ant build on the file build.xml with the target<br> "test" (either from a command line or inside Eclipse).</li><br></ul>
| + | |
− | | + | |
− | <p>The above assumes that ant or Eclipse are installed, that<br>JAVA_HOME is set to Java 6.0 or higher and that you have a direct<br>connection to the Internet (no proxy server in between). Note, the build<br>downloads Buckminster using the Eclipse director, the director has<br>sometimes trouble with certain java 6 versions. So if you get 'Unpack<br>facility not configured' errors then check out the troubleshooting<br>section below. After some minutes you will see a "build" folder (in the<br>Eclipse workspace) with a some of subfolders:</p><br><ul><br> <li>tools: Contains the tools needed for the build. There is a<br> subfolder for the Buckminster Director tool, that is used to download<br> Buckminster and a subfolder for the Buckminster Eclipse installation<br> that is used to perform the actual build.</li><br> <li>result: Contains the build result and all needed sources and<br> binary dependencies.</li><br> <li>result/workspace: Contains the Graphiti sources (plugin,<br> features, etc.) to be build.</li><br> <li>result/tp: The Eclipse target installation used by Buckminster<br> to build against.</li><br> <li>result/junit-workspace: The Eclipse workspace that was used to<br> run the Graphiti tests.</li><br> <li>result/temp: A temporary folder used by Buckminster.</li><br></ul><br><p>The tools are downloaded as part of the build. The build<br>workspace (result/workspace) and test workspace (result/junit-workspace)<br>can be opened in Eclipse as workspaces.</p>
| + | |
− | | + | |
− | <h4>Understanding Buckminster</h4><br><p>This section may help to make it a bit clearer how Buckminster<br>works. The main functionality (for use within the Graphiti build):</p><br><ul><br> <li>Buckminster creates workspaces and target platforms for<br> running the build and test. These workspaces are the same or similar to<br> what you see in your IDE.</li><br> <li>Buckminster uses the includes and dependency definitions in<br> the MANIFEST.MF (for a plugin project) and the feature.xml (for a<br> feature project) to download related projects or binaries. Additional<br> dependencies are defined in a buckminster.cspex file. The build and<br> buckminster files only refer to so-called root feature projects. The<br> includes and dependencies of those projects then automatically drive<br> the download of all other relevant projects and binaries into the<br> workspace and target platform.</li><br> <li>Buckminster can search many types of locations for<br> dependencies. The locations and their search order are defined in a<br> configuration file (the graphiti.rmap files in the build and test<br> subfolders). Development projects are downloaded from cvs while binary<br> dependencies are coming from update sites.</li><br> <li>When downloading dependencies, Buckminster determines which<br> dependency to place in the workspace (as a development project) and<br> which dependencies to place in the target platform. The choice where to<br> place what is defined in the graphiti.rmap file using an attribute:<br> source="true", which signals a dependency to be a source (so should be<br> placed in the workspace).</li><br> <li>Buckminster has several commands which can be executed against<br> a workspace/target platform, for example run tests, build a p2.site<br> etc.</li><br></ul>
| + | |
− | | + | |
− | <p>A short description on the meaning of some files in the Graphiti<br>build:</p><br><dl><br> <dt><b>rmap file</b></dt><br> <dd>Defines where and how to download dependencies and development<br> projects, contains cvs locations, update site url's and specifies an<br> order in which these locations are searched. Also identifies which<br> sources are considered to contain sources (i.e. development projects)<br> and which are used for binary dependencies.</dd><br> <dt><b>cquery file</b></dt><br> <dd>Defines which rmap file to use and defines the root (feature)<br> project, the content (defined via the feature.xml) and the dependencies<br> of the content are downloaded.</dd><br> <dt><b>mspec file</b></dt><br> <dd>This file is the starting point. It specifies which cquery<br> file to use and also how the plugins/development projects downloaded as<br> part of resolving the cquery file should be treated, or materialized.<br> Materialized means: where to place the downloaded plugins/development<br> projects, in the workspace or in the target platform.</dd><br></dl>
| + | |
− | | + | |
− | <h4>Main Build Steps</h4><br><p>The Graphiti build process is executed through an ant script<br>which calls Buckminster for the main build steps. The ant script goes<br>through the following steps.</p><br><ol><br> <li>Download the p2 director and using the director, download<br> buckminster</li><br> <li>Download the development projects from CVS and use them to<br> create a workspace with a target platform</li><br> <li>Download/retrieve the dependencies and set them in the target<br> platform</li><br> <li>Build the software (including the test plugins)</li><br> <li>Build the update site</li><br> <li>Create downloadable zips</li><br> <li>Run the test cases and convert the test results to a format<br> which can be read by Hudson</li><br></ol>
| + | |
− | | + | |
− | <p>The following sections describe the necessary files for the<br>build: the ant xml files and the buckminster files.</p>
| + | |
− | | + | |
− | <h4>Structure of build projects/features</h4><br><p>The build infrastructure for Graphiti is centered around a releng<br>project containing the ant build.xml file and the main Buckminster<br>files. A number of other feature projects are used to define which<br>plugins are part of the build, part of the update site and which are<br>part of the tests. The releng and feature projects can be found in the<br>CVS location<br>dev.eclipse.org/cvsroot/modeling/org.eclipse.gmp/org.eclipse.gmp.graphiti<br>in the folders releng (releng project) and features (feature projects).</p>
| + | |
− | | + | |
− | <p>The org.eclipse.graphiti.releng project contains the ant and the<br>buckminster files for the build and test steps. The content of this<br>project and the buckminster setup is described in more detail below.<br>This project is the most important one to look at when trying to<br>understand Graphiti's build setup. The feature projects are used to<br>structure the output of the build in the correct way, i.e. which<br>features/plugins are made available via an update site, which<br>features/plugins are involved in testing etc. Here is a summary of these<br>feature projects:</p><br><ul><br> <li><b>org.eclipse.graphiti.site</b> - Defines the update site and<br> the main features. This project contains a buckminster.cspex file to<br> define extra dependencies for stuff added via package imports (this is<br> described in more detail below). The included features are:<br> <ul><br> <li>org.eclipse.graphiti.feature - The Graphiti runtime feature</li><br> <li>org.eclipse.graphiti.feature.examples - The documentation and<br> the tutorial plugin for Graphiti</li><br> </ul><br> <li><b>org.eclipse.graphiti.tests-feature</b> - Contains the test<br> plugins and their dependencies. The test plugins contain test launch<br> configurations (see below running test cases).</li><br></ul>
| + | |
− | | + | |
− | <h4>Main build project: org.eclipse.graphiti.releng</h4><br><p>After downloading the main build project from cvs you can see<br>that it consists of several files in the root and the xsl subfolder.<br>Here is a summary for the meaning of different files:</p><br><ul><br> <li><b>build.properties:</b> defines several properties related to<br> how to sign plugins and where to download the director and buckminster<br> applications. Used within the central build.</li><br> <li><b>local.properties:</b> defines several properties related to<br> how to sign plugins and where to download the director and buckminster<br> applications. Used within the local build.</li><br> <li><b>build.xml:</b> the main build file which drives the build<br> and test phases. Its content is described in more detail below.</li><br></ul><br><p>The Graphiti build consists of three steps: 1) build the Graphiti<br>plugins and features, 2) use these build results to set up a P2 update<br>site and 3) run the tests. The build.xml file has 3 main targets:</p><br><ul><br> <li>test: is the main target, builds the core plugins, creates an<br> update site and runs the test cases</li><br> <li>promote.sites: copies the created update site to another<br> location where it is available for everyone, again uses Buckminster to<br> do so</li><br> <li>clean.all: cleans the tools and output directories, uses<br> standard ant remove directory commands</li><br></ul><br><p>The interesting one is the test command, this one is explained in<br>more detail.</p>
| + | |
− | | + | |
− | <p>This test target depends on the target site.p2 and triggers the<br>test runs for Graphiti (there are 3 of them: a JUnit test run for each<br>the plugins org.eclipse.graphiti and org.eclipse.graphiti.ui and a<br>SWTBot-based test).</p>
| + | |
− | | + | |
− | <p>Before the site.p2 command is run which triggers a build using<br>Buckminster, creates an update site again by using Buckminster and<br>providing human readable content descriptions.</p>
| + | |
− | | + | |
− | <p>This step again depends on the "provision" step which sets up all<br>the relevant variables (e.g. from the properties files), installs the<br>Buckminster Director application, the actual Buckminster installation<br>and collects the target platform and workspace to do the build.</p>
| + | |
− | | + | |
− | <h5><b>Build:</b></h5><br><p>The build setup consists of the following files:</p><br><ul><br> <li><b>build.cquery:</b> this files defines the root request<br> project. It is the releng project that again defines the dependencies<br> which projects to retrieve from cvs; this is mainly the site feature<br> with its complete dependency tree. The content of the root feature<br> project in the feature.xml (and an additional dependency definition<br> file, a cspex file, see below) specifies which development projects to<br> download.</li><br> <li><b>build.mspec:</b> this file specifies that dependencies<br> which are source (source="true") should be materialized in the<br> workspace, while the other dependencies are materialized in the target<br> platform.</li><br> <li><b>build.rmap:</b> defines where and how development projects<br> and dependencies can be downloaded. This file for example lists cvs<br> locations and several update sites.</li><br></ul><br><p>The mspec file is the starting point for Buckminster. Its<br>content:</p><br><pre><br> <code><br>&lt;?xml version="1.0" encoding="UTF-8"?&gt;<br>&lt;mspec:mspec xmlns:mspec="http://www.eclipse.org/buckminster/MetaData-1.0" installLocation="" materializer="p2" name="build.mspec" url="build.cquery"&gt;<br> &lt;mspec:property key="target.os" value="*"/&gt;<br> &lt;mspec:property key="target.ws" value="*"/&gt;<br> &lt;mspec:property key="target.arch" value="*"/&gt;<br> &lt;mspec:mspecNode materializer="workspace" filter="(buckminster.source=true)"/&gt;<br>&lt;/mspec:mspec&gt;<br> </code><br></pre>
| + | |
− | | + | |
− | <p>This file specifies two things: firstly, which query file to use,<br>the cquery file is explained below, it defines which root feature<br>project to download as a starting point for the build and secondly, how<br>to materialize: as a default materialize to the target platform<br>(materializer="p2"), only if source is "true" materialize to the<br>workspace. As noted above the mspec file defines which cquery file to<br>use; here's the content of the cquery file:</p><br><pre><br> <code><br>&lt;?xml version="1.0" encoding="UTF-8"?&gt;<br>&lt;cq:componentQuery xmlns:cq="http://www.eclipse.org/buckminster/CQuery-1.0" resourceMap="build.rmap"&gt;<br> &lt;cq:rootRequest name="org.eclipse.graphiti.releng" componentType="buckminster"/&gt;<br> &lt;cq:advisorNode namePattern="^org\.eclipse\.graphiti(\..+)?\.source*" skipComponent="true"/&gt;<br> &lt;cq:advisorNode namePattern=".*" useMaterialization="false" useTargetPlatform="false"/&gt;<br>&lt;/cq:componentQuery&gt;<br> </code><br></pre><br><p>The cquery file defines which rmap file to use, the rmap file<br>specifies the download locations (update sites) and cvs locations to<br>download dependencies. Buckminster needs a root project to use as a<br>starting point which is defined by the rootRequest tag. The advisorNode<br>tag is used to tell Buckminster to ignore dependencies (defined in<br>feature.xml files) which have a pattern starting with<br>org.eclipse.graphiti and ending with source. This is needed because the<br>SDK feature projects contain these dependencies. However, these *.source<br>plugins/features do not exist in cvs but are created by<br>Buckminster/Eclipse during the build. So they should be ignored when<br>downloading dependencies. See <a<br> href="http://www.eclipse.org/forums/index.php?t=tree&th=162136&">here</a><br>for a thread on the Buckminster newsgroup on this.</p>
| + | |
− | | + | |
− | <p>The rmap file describes where Buckminster can download the<br>development projects and dependencies. For example the following snippet<br>defines the cvs location for the Graphiti plugins (note also the<br>source="true" attribute which signals Buckminster to treat these<br>dependencies as source and put them in the workspace (see the mspec file<br>above)).</p><br><pre><br> <code><br> &lt;rm:searchPath name="releng"&gt;<br> &lt;rm:provider componentTypes="buckminster,osgi.bundle" readerType="cvs"&gt;<br> &lt;rm:uri format="{0},org.eclipse.gmp/org.eclipse.gmp.graphiti/releng/{1}"&gt;<br> &lt;bc:propertyRef key="cvs.repository"/&gt;<br> &lt;bc:propertyRef key="buckminster.component"/&gt;<br> &lt;/rm:uri&gt;<br> &lt;/rm:provider&gt;<br> &lt;rm:provider componentTypes="eclipse.feature" readerType="cvs"&gt;<br> &lt;rm:uri format="{0},org.eclipse.gmp/org.eclipse.gmp.graphiti/features/{1}"&gt;<br> &lt;bc:propertyRef key="cvs.repository"/&gt;<br> &lt;bc:propertyRef key="buckminster.component"/&gt;<br> &lt;/rm:uri&gt;<br> &lt;/rm:provider&gt;<br> &lt;/rm:searchPath&gt;<br> </code><br></pre>
| + | |
− | | + | |
− | <p>Then create the p2 site, note that the top feature used for the<br>site is not included in the site (as a default, see an option a few<br>lines further down).</p><br><pre><br> <code><br>&lt;buckminster command="perform" workspace="${workspacePath}"&gt;<br> &lt;cmdargs&gt;<br> &lt;arg value="org.eclipse.graphiti.site#site.p2" /&gt;<br> &lt;/cmdargs&gt;<br>&lt;/buckminster&gt;<br> </code><br></pre>
| + | |
− | | + | |
− | <p>This concludes the build step. The next step is testing using the<br>output from the build process.</p>
| + | |
− | | + | |
− | <h5>Test:</h5><br><p>The test task finally call the tests on the previously build<br>plugins.</p>
| + | |
− | | + | |
− | <p>The build.xml has a "test" target which performs the necessary<br>steps to run the test cases. The emma command will run junit with code<br>coverage checking. See a later section on code coverage setup. Instead<br>of the emma command, the command 'junit' could also have been used (but<br>then no code coverage report is created of course). The junit test run<br>itself consists of calling a launch configuration and defining the<br>output of the test results. The launch configuration has been created<br>manually and checked into cvs.</p><br><pre><br> <code><br>&lt;buckminster command="emma" workspace="${workspacePath}"&gt;<br> &lt;cmdargs&gt;<br> &lt;arg value="-l" /&gt;<br> &lt;arg value="org.eclipse.graphiti.tests/AllJunitTests.launch" /&gt;<br> &lt;arg value="-o" /&gt;<br> &lt;arg value="${testResultsPath}/output/JUnit-graphiti-results.xml" /&gt;<br> &lt;arg value="--stdout" /&gt;<br> &lt;arg value="${testResultsPath}/output/JUnit-graphiti-stdout.txt" /&gt;<br> &lt;arg value="--stderr" /&gt;<br> &lt;arg value="${testResultsPath}/output/JUnit-graphiti-stderr.txt" /&gt;<br> &lt;arg value="--xml" /&gt;<br> &lt;arg value="${testResultsPath}/output/JUnit-graphiti-coverageReport.xml" /&gt;<br> &lt;arg value="--flatXML"/&gt;<br> &lt;/cmdargs&gt;<br>&lt;/buckminster&gt;<br> </code><br></pre><br><p>The flatXML argument is used to convert the output from the tests<br>to a result which can be read by Hudson.</p>
| + | |
− | | + | |
− | <h4>Defining dependencies: buckminster.cspex</h4><br><p>Buckminster uses the explicit plugin dependencies in the<br>MANIFEST.MF to download related project and binary dependencies.<br>Buckminster can however not directly/automatically resolve dependencies<br>defined by the import-packages directive. This is logical as Buckminster<br>can not identify the plugins to download, just on the basis of the<br>package name. To solve this, it is possible to define additional<br>dependencies using a so-called buckminster.cspex file. The Graphiti<br>build has a buckminster.cspex file in two feature projects and in the<br>releng project:</p><br><ul><br> <li>org.eclipse.graphiti.site-feature</li><br> <li>org.eclipse.graphiti.tests-feature</li><br> <li>org.eclipse.graphiti.releng</li><br></ul><br><p>The releng project is used in therootRequest of the cquery file<br>and delegates to the two feature projects. Note, the file containing the<br>additional dependencies must be called buckminster.cspex.</p><br><p>This cspex file defines dependencies on several eclipse features.<br>The rmap file define where to find these features (i.e. which update<br>site for example).</p>
| + | |
− | | + | |
− | <h4>Testing</h4><br><p>Testcases are run through Buckminster using a launch<br>configuration (so Buckminster calls a launch configuration). Graphiti<br>has 3 launch configurations one for running JUnit tests for the core<br>plugin (org.eclipse.graphiti), one for running JUnit tests for the UI<br>plugin (org.eclipse.graphiti.ui) and one for running the SWTBot tests<br>for the complete framework (UI tests).</p><br><p>The launch configuration is easy to create, after running your<br>test suite in Eclipse, a run configuration should be present in the<br>window Run > Run Configurations. To store the run/launch configuration<br>in a file: in the last tab of the configuration (Common), the save as<br>location is set to a shared file in a project. The file is checked-in<br>into cvs. This shared file is then used as the input to the test run<br>through buckminster, see the call for running the tests above. The<br>output location is specified with the -o parameter. The xml in the<br>output location is used by Hudson to create a test report.</p>
| + | |
− | | + | |
− | <h4>Defining and creating an update site</h4><br><p>The Graphiti build creates an update site, which site is defined<br>through a feature project: the org.eclipse.graphiti.site project. It has<br>one important file:</p><br><ul><br> <li>feature.xml: defines which features are part of the update<br> site</li><br></ul><br><p>The feature.xml contains 2 features:</p><br><ul><br> <li>org.eclipse.graphiti.feature</li><br> <li>org.eclipse.graphiti.feature.examples</li><br></ul><br><p>To build the update site the build xml file contains the<br>following snippet:</p><br><pre><br> <code><br>&lt;buckminster command="perform" workspace="${workspacePath}"><br> &lt;cmdargs><br> &lt;arg value="org.eclipse.graphiti.site:eclipse.feature#site.p2" /><br> &lt;/cmdargs><br>&lt;/buckminster><br> </code><br></pre><br><p>Buckminster is called with the org.eclipse.graphiti.site feature<br>(note feature id is used here, not the project name - here thi sis not<br>relevant since they are the same, but they might differ), and the<br>site.p2 call. As a default (is option controlled) the content of the<br>passed in feature is used for the content of the site, so the feature<br>project (org.eclipse.graphiti.site) itself is not part of the update<br>site.</p>
| + | |
− | | + | |
− | <h4>Code Coverage</h4><br><p>It is quite easy to enable code coverage analysis as part of the<br>test run. This section explains how to set it up.</p><br><p>First make sure that the<br>org.eclipse.buckminster.emma.headless.feature.feature.group feature is<br>installed as part of the Buckminster install. See the<br>install.buckminster target in Graphiti's build.xml. To enable code<br>coverage analysis in Buckminster you have to use the 'emma' command<br>instead of the 'junit' command when calling Buckminster and pass an<br>extra xml parameter with the output location of the code coverage<br>analysis.</p>
| + | |
− | | + | |
− | <br>
| + | |
− | | + | |
− | <br>
| + | |
The information on this page was completely outdated and thus removed.