PTP/release notes/2.1

What's new in PTP 2.1

• Resource Managers
  • New more flexible resource manager for Open MPI. This resource manager does not require a proxy agent to be running on the target system, and can be use as a template to target other MPI implementations. It also works with Open MPI 1.2 and 1.3.
  • Improvements to the Parallel Environment resource manager, as well as support for the new debugger.
• Parallel Debugger
  • Removed the dependency on MPI. This allows the debugger to work with virtually any MPI implementation.
  • Flexible architecture allows different routing and communication layers
  • Full asynchronous command support
• Parallel Language Development Tools (PLDT)
  • Unified Parallel C (UPC) support
  • MPI code templates
• Parallel Tools Framework (formerly the Performance Tools Framework)
  • Added support for PAPI Component counter selection
  • Added parametric analysis capabilities (Supported primarily for TAU at this time)
  • Significant modifications to increase work-flow flexibility
• Remote Development Tools (RDT)
  • Enables C and C++ projects to be located on a remote machine
  • Provides remote indexing and parsing services
  • New remote C/C++ project wizard
  • Automatic source code delta handling (the index is automatically updated when files in your project are added/removed/changed)
  • Remote "scanner info" support to allow the user to define include paths and defined preprocessor macros as a context for the parser to operate
  • Remote Search, Call Hierarchy, Navigation (e.g. Go To Declaration), Content Assist, Type Hierarchy
  • Remote Standard Make for building remote makefile-based projects
• Services Model
  • Allows the user to configure which services in their system are mapped to which providers and locations (e.g. to a remote machine).
  • Allows ISVs and tool implementers to define new services, and contribute providers to any defined services.
• Cell/B.E IDE Contribution
  • Custom source code templates for Cell development
  • Full configurable build properties for PPU and SPU using GNU and XL compilers
  • Managed Build for PPU and SPU using GNU and XL compilers
  • Remote launch and debug of Cell Applications
  • Support for PPU & SPU combined remote debugger
  • Cell performance tools support
  • ALF programming model support
  • IBM PDT (Performance Debug Tools) instrumentation plugins..
  • Mambo Simulator plugins
  • Five pre-configured Cell projects

Prerequisites

Component	OS	Java	Eclipse	CDT	RSE	MPI	gdb
PTP	Linux Mac OS X Windows (remote only)	1.5 or later	3.4.1 3.3 (limited)	5.0.1 4.0.x (limited)	3.0	Open MPI 1.2.x or 1.3.x MPICH 2 IBM Parallel Environment	6.3 - 6.8
RDT	Linux Mac OS X Windows	1.5 or later	3.4.1	5.0.2	3.0	N/A	N/A

Installation

You will need to install Java, Open MPI, Eclipse, CDT (C/C++ Development Tools), and PTP, in that order. You may then need to build a small platform-specific set of runtime components. The steps are detailed below:

Install Java

Java requirement: If you are using the Linux operating system, download and install either the Sun Java Runtime Environment or the IBM Java Runtime Environment. The version of Java that ships with Linux (gcj) will not work with Eclipse. See the Trouble Shooting section for more information.

Install Open MPI

You may already have Open MPI installed. Check for this and the version with the following command:

       ompi_info | grep 'Open MPI:'

If the version is not correct, or the command is not found, download and unpack a recent version of Open MPI 1.2 or 1.3. (e.g. 1.2.7 or 1.3b1)

Configure, build and install from source

In the main Open MPI source directory, run the configure command (some platforms may require additional options, such as --disable-pty-support for MacOS X - see the Open MPI documentation for more information):

       ./configure

Build and install Open MPI using:

       make 
       make install

Note that if you install Open MPI in a nonstandard location, you may need to build PTP differently. See the troubleshooting section below. Make sure that your PATH and LD_LIBRARY_PATH point to appropriate places if necessary also.

ALWAYS test your MPI installation by running a small MPI job, e.g.

       mpicc test.c -o test
       mpirun -np 4 test

Install Eclipse

Install Eclipse 3.4 and CDT 5.0.1 or later, for your architecture. Although PTP can be used in certain limited applications with Eclipse 3.3, only Eclipse 3.4 will be described here.

Note that RDT requires CDT 5.0.2. Since 5.0.2 is not currently released, you will need to get the latest nightly build and follow the installation instructions below.

You have two choices for Eclipse installation:

1. Install the full Eclipse SDK ("Classic") followed by CDT - if you plan to do Java or Eclipse development in addition to C/C++, you'll need this.
2. (easiest, lightweight, smaller footprint) Install Eclipse IDE for C/C++ development (available from Ganymede downloads page, includes CDT 5.0.1). - if you only plan to do C/C++ development, this is all you need. This is about 1/3 to 1/4 the size of the full SDK, and doesn't include the Java or Eclipse Plug-in development parts, which I assume you don't need. The UI is simpler since you don't have all those additional options.

(1) Full Eclipse install

1. Go to Eclipse downloads and download "Eclipse classic" for your platform. This is the full SDK.

(2) Lightweight install for just C/C++ development

1. Go to Eclipse downloads and download "Eclipse IDE for C/C++ Developers." This is the basics of the Eclipse Platform, plus CDT, already packaged together.
2. Note that this is the bare bones install, does not include the UPC support etc. Read on for adding this support.

Unpack and launch Eclipse

After downloading and extracting Eclipse (zip or tar file), there's nothing else to the installation process from the Eclipse side. In the extracted folder, simply run the eclipse executable file. It will ask for a workspace location; the default is fine.

Install the C/C++ Development Toolkit (CDT)

CDT adds C/C++ tools to your base Eclipse distribution, or if you installed the "Eclipse IDE for C/C++ Developers" above, you have the opportunity to install additional tools, e.g. UPC tools.

There are two methods for installing CDT, depending if you are installing 5.0.1 or the unreleased 5.0.2 version.

Installing CDT 5.0.1

Installation of CDT 5.0.1 is via the Ganymede update site that is already pre-installed into your Eclipse distribution:

1. In Eclipse, select Help > Software Updates...
2. At top, make sure the "Available Software" tab is selected.
3. Open the "Ganymede Update Site" item.
4. Select the "C and C++ Development" checkbox.
5. Click "Install..." and follow the instructions.
6. When asked, click "Yes" to restart the Eclipse workbench.

Installing CDT 5.0.2 (pre-release) - Required for RDT

Pre-release versions of CDT are installed using the update manager, but you first need to download a copy of the update site to you local machine.

1. Go to the nightly builds page and click on the latest build.
2. Download "cdt-master-5.0.2-IYYYYMMDDHHMM.zip".
3. In Eclipse, select Help > Software Updates...
4. At top, make sure the "Available Software" tab is selected.
5. Click "Add Site..." button then click the "Archive..." button.
6. Locate the "cdt-master-5.0.2-IYYYYMMDDHHMM.zip" file you downloaded in the first step and click OK.
7. You should now see a new update site that includes the name of the zip file.
8. Expand the site and select the features you wish to install.
   • Just select them all if you are unsure. You might want to uncheck "Eclipse CDT Testing Feature" under "CDT Optional Features" - it can add an extra unneeded page to wizards.
   • If you don't need any optional features, just select "Eclipse C/C++ Development Tools". Select the "SDK" version if you're planning to develop plugins that use CDT.
   • If you already installed the "Eclipse IDE for C/C++ Developers" then the basic features are already installed, and you might only see one item under "CDT Main Features."
9. Click "Install..." and follow the instructions.
10. When asked, click "Yes" to restart the Eclipse workbench.

Install Remote System Explorer (RSE)

If you plan to use the Remote Development Tools (RDT), or wish to use PTP with the Remote System Explorer (RSE), it is recommended that you install it manually. Although the update manager should automatically install RSE if required, it is not always successful.

You can do this from Eclipse as follows:

1. Open Help > Software Updates...
2. At top, click on "Available Software" tab.
3. Open the "Ganymede Update Site" item.
4. Open the "Remote Access and Device Development" item.
5. Select "Remote System Explorer End-User Runtime" item.
6. Click "Install..." and follow the instructions.
7. When asked, click "Yes" to restart the Eclipse workbench.

Install PTP and RDT

Installation of PTP is in two parts. The first part comprises installing PTP into your Eclipse distribution on the local machine. The second part comprises installing optional components on the target machine you wish to use with PTP or RDT. This target machine could be the same as that running Eclipse, or a remote system.

Installing PTP locally

You can do this from Eclipse as follows:

1. Open Help > Software Updates...
2. At top, click on "Available Software" tab.
3. Click "Add Site..." button then enter "http://download.eclipse.org/tools/ptp/releases/2.1" in the text field. Click OK.
4. Expand the site you just added, and select the features you wish to install.
   • Just select them all if you are unsure.
   • Select "Parallel Tools Platform 2.1" if you just want PTP.
   • Select "Remote Development Tools" if you just want RDT.
5. Click the "Install..." button and follow the instructions.
6. When asked, click "Yes" to restart the Eclipse workbench.

PTP features

These descriptions refer to the 2.0.2 feature names. Need to update to 2.1

1. Expand the "PTP Update site" item, expand "Parallel Tools Platform 2.0" and check the features you want:
   • Parallel Tools Platform End-User Runtime (required)
   • PTP Parallel Language Development Tools (optional, but recommended)
   • PTP Common External Components (required, platform-specific fragments, proxy & util libraries)
   • PTP Remote Tools Enabler (optional for remote, must have PTP Remote Tools installed)
   • PTP RSE Enabler (optional for remote, must have RSE installed )
   • PTP Support for Open MPI (optional, required to use Open MPI as Resource Manager)
   • PTP Support for IBM Parallel Environment (optional, required to use IBM PE as Resource Manager)
   • PTP Support for IBM LoadLeveler (optional, required to use IBM LL as Resource Manager)
   • PTP Remote Tools (optional)

Installation Examples

Note: these lists are in terms of 2.0.2 features, the names may have changed. NEED TO UPDATE THIS.

User wishes to use PTP with the parallel language development tools, and use Remote Tools for remote access. Parallel jobs will be submitted using OpenMPI. Select the following components:

• Parallel Tools Platform End-User Runtime
• PTP Parallel Language Development Tools
• PTP Common External Components
• PTP Remote Tools Enabler
• PTP Support for Open MPI
• PTP Remote Tools

User wishes to use PTP directly on a machine with LoadLeveler installed (no remote services required). PLDT is not required. Select the following components:

• Parallel Tools Platform End-User Runtime
• PTP Common External Components
• PTP Support for IBM LoadLeveler

Users wishes to use PTP with a remote machine that has IBM PE installed. PLDT is required. RSE has already been installed in Eclipse. Select the following components:

• Parallel Tools Platform End-User Runtime
• PTP Parallel Language Development Tools
• PTP Common External Components
• PTP RSE Enabler

User wishes to use Remote Tools only (without PTP). Select the following component:

• PTP Remote Tools

Remote Proxy Agent Installation

If you wish to control a resource manager running on a remote system, you will need to install the corresponding proxy agent on the remote machine.

THIS IS INCORRECT FOR 2.1. NEED TO UPDATE.

1. Download the resource manager proxy agents onto the remote machine from http://eclipse.org/ptp/downloads.php
2. Untar (or unzip) the file.
3. Follow the instructions below for building the proxy agents.

Building Proxy Agents

For local resource managers (same machine), this procedure should still be ok for 2.1 (change versions numbers to 2.1.0 etc.)

Locate the plug-in org.eclipse.ptp.os.arch_x.y.z.###, where os and arch correspond to your operating system and architecture respectively, x.y.z corresponds to the release version of that particular plugin (e.g. 2.0.0), and ### corresponds to the build date (e.g. 200804010706). The plugin will either be in your Eclipse installation plugins directory if you installed the resource managers locally, or in the location you unzipped the proxy agents in the previous step.

For example,

 cd  <eclipse-install-location>/eclipse/plugins/org.eclipse.ptp.linux.x86_2.0.0.200804010706

After changing to this directory, run the build command:

   sh BUILD

This should build and install the executables. Check the bin directory to see if there are executables in it (e.g. ptp_orte_proxy and sdm). If you see any errors here, please refer to the Trouble Shooting section below. If you installed Open MPI in a non-standard location, see below also for an alternate method of building Open MPI.

Note: be sure that LD_LIBRARY_PATH is set (to point to OpenMPI libs, e.g. /usr/local/lib) for the environment in which eclipse is launched, and the environment in which the above build is performed. PATH should point to mpi as well, e.g. /usr/local/bin.

When you create a resource manager, you will need to navigate to the bin directory and select the appropriate resource manager proxy agent. When you configure a debug launch configuration, you will need to select the sdm executable in this directory.

Test your installation

Start Eclipse. (You may need to use the -clean parameter to force it to recognize new features and plug-ins).

Create e.g. a small MPI C project. Open the PTP Runtime perspective (Window>Open Perspective>Other...). The runtime service will start and you should see a machine and one or more nodes appear in the 'Machines View'. Add a resource manager and start it. Create a launch configuration (Run -> Open Run Dialog...) and create a launch configuration for a Parallel Application.

More details on using PTP can be found in Help>Help Contents. Click on "PTP: Parallel Tools Platform ". The PTP help is also available online.

Known Issues

Resource Manager

1. There are synchronization issues when two (or more) remote resource managers attempt to start simultaneously. This only happens when PTP is first launched, and attempts to reconnect to resource managers that were running when Eclipse was closed.

Reproduce By: Start two remote resource managers that use remote tools connections. Exit from Eclipse. Restart Eclipse.

Workaround: Shutdown resource managers before exiting Eclipse

2. If two resource managers share the same remote connection, stopping one will also stop the other.

Reproduce By: Create two resource managers that use the same remote connection. Start both resource managers. Stop one resource manager.

Workaround: None. This is expected behavior since shutting down a resource manager also closes the connection.

Debugger

1. Accelerator keys (F5, F6) operate on the current process selected in Debug view. There is currently no way to use these keys for the Parallel Debug view.

Launch configurations

1. Certain sequences can result in a blank Resources page in the Resources tab.

Reproduce By: Start a resource manager. Open launch configuration and choose resource manager from Main tab. Select Resources tab and fill in some resources. Close launch configuration and shutdown resource manager. Start a different resource manager. Open the same launch configuration and switch to Resources tab, then switch back to main tab. Set the launch config to new the new resource manager, then switch to resources.

Workaround: None.

Troubleshooting

1. I get lots of error messages about missing include files when running the BUILD script.
   • This is because OpenMPI was not configured to include the development headers. You need to re-run the configure command for the OpenMPI installation specifying the --with-devel-headers option.
2. I get errors like "configure: error: utils directory not found" when running the BUILD script.
   • Make sure that you don't have old versions of PTP along side the version you're building. The BUILD script will get confused if there is more than one version of PTP. (i.e. remove any older versions from your plugins directory)
3. I installed OpenMPI in a non-standard location. Do I need to do anything differently?
   • Yes. The standard location is /usr/local. If you install it to, e.g., /usr/local/openmpi, then when you build the platform-specific PTP code, instead of sh BUILD you should use the following command: CFLAGS=-I/usr/local/openmpi/include LDFLAGS="-L/usr/local/openmpi/lib -Wl,-rpath,/usr/local/openmpi/lib" sh BUILD
4. Note: The application must be able to be run from the command line without supplying additional arguments (such as -x). In particular, the PATH and LD_LIBRARY_PATH must be set correctly on the frontend and all the nodes. This limitation will be removed in PTP 2.1.
   • Unfortunately the current (2.0) version of PTP does not allow you to alter the mpirun command line (in order to add '-x LD_LIBRARY_PATH', for example), so the only way to get around this problem is to add the path to the libpetsc.so library to the LD_LIBRARY_PATH variable in your .bashrc. The next version of PTP (2.1, due around August) will remove this restriction.
   • The debugger does not handle this failure condition very well. We'll be looking into improving the error messages in 2.1.
5. Note: The debugger has some issues when used on C++ code. We'll determine what these problems are and make sure they're fixed in 2.1.

See also the older 1.1.1 Release notes troubleshooting section

FAQ

1. Simulating multiple nodes with OpenMPI: If you are testing on a single node, edit <openmpi_install>/etc/openmpi-default-hostfile (default is /usr/local/etc/openmpi-default-hostfile) and add lines containing 'node0', 'node1', etc. Edit /etc/hosts and add a localhost entry for each name you added to the openmpi-default-hostfile. So, for 'node0', add '127.0.0.1 node0' to /etc/hosts. This will simulate a multi-node machine.
2. How do I debug my program remotely?
   • Currently you must move the executable to the remote target(s) yourself. The run/debug launch configuration contains an entry for the location, on the remote machine, of the executable to run/debug. The source in your local machine (running eclipse and PTP) must match the executable on the remote machine for source-level debugging to work correctly. This will change when CDT more completely handles remote resources, hopefully by CDT 5.0 (summer '08)
     • The remote facilities in PTP 2.1 can help with this... need to test and document here ...
3. How do I launch eclipse?
   • Normally just run the eclipse executable - however, you may want to do this from a command line instead of from a "shortcut" - some OS's or windowing systems do not properly send the environment information. To make sure the PATH and LD_LIBRARY_PATH get correctly sent to the eclipse enviornment, you may want to open a shell and run "./eclipse" -- See also the troubleshooting section of the PTP 1.1.1 release notes for more information.