PTP/designs/SCI

Introduction

SCI (Scalable Communication Infrastructure) is a light-weight communication library which provides scalable message transmission functions for a client-server model, especially for a central server associated with a large number of clients. Internally, SCI makes use of a classical tree-based hierarchical structure to build up message transmission paths among server and clients. Typically, the server can be considered as front end and the clients can be considered as back ends.

Installation

Get the source code from the Eclipse CVS repository or download it into your Eclipse workspace. To download the SCI source from the Eclipse CVS repository:

• CD to a directory where the SCI source will be extracted
• Set CVSROOT by issuing the command export CVSROOT=:pserver:anonymous@dev.eclipse.org/cvsroot/tools
• Checkout the SCI source by issuing the command cvs checkout org.eclipse.ptp/tools/sci/org.eclipse.ptp.sci
• The SCI source will be located in the org.eclipse.ptp/tools/sci/org.eclipse.ptp.sci subdirectory
• Note that you can specify qualifiers to the cvs command to extract SCI source other than at the HEAD (latest) level.

To download the SCI source into your Eclipse workspace:

• Start Eclipse
• Open the Eclipse installation wizard by clicking the Help menu then clicking Install New Software.
• To download the initial SCI source, select the latest Eclipse release download site, Helios in 2010, from the Work with: dropdown. To download SCI updates, select the Eclipse Project Update site
• Open the General Purpose Tools node in the software list and check the checkbox next to PTP Scalable Communication Infrastructure (SCI)
• Click Next and follow the remaining prompts in the installation wizard
• The SCI source code will be installed in the plugins directory of your Eclipse installation. The installation process will create a subdirectory where the directory name includes a time stamp. For instance, org.eclipse.ptp.sci_1.0.0.201006142322.
• Once you download the SCI source, you must transfer the entire contents of this directory to the system where you will build SCI using FTP or other file transfer mechanism.

1. For source code, in SCI’s root directory, do

./configure
make
make install

If you want to enable the OpenSSL security mechanism in SCI, an option --enable-openssl can be specified:

./configure --enable-openssl

2. Launch scid: Assume SCI is installed into the directory /opt/sci, scid is located in /opt/sci/sbin. You must have root privileges to start scid. Once you have root privileges, start scid as /opt/sci/sbin/scid. You can also modify your system startup scripts to start scid at system startup.

Topology

Typically, a SCI session contains the following processes:

• A front end process (FE).
• One or multiple back end processes (BEs).

If using standalone agent mode which will be explained below, there will also be

• zero or multiple agent processes (scia).

The processes build up a tree-based structure. The front end is the tree root and the back ends are the leaves. The communications are between the front end and the back ends. The messages are forwarded by agents/embedded agents and messages also can be filtered by the plug-ins running in front end or agents/embedded agents when they are passing messages either upstream or downstream.

SCI supports both stand-alone agent mode and embedded agent mode which can be specified by the environment variable SCI_EMBED_AGENT=[yes|no]. The interfaces for both modes are almost identical and are transparent to users with the exception of a connection call-back function which can be only used in the embedded mode. For stand-alone mode, there are scia processes which help to forward messages while for the embedded mode, the scias are embedded into the user’s back end which are called EA here. The tree-based hierarchical structures are the same for both modes. The front end is actually the root agent.

Stand-alone agent mode

Embedded agent mode

Note: a big block stands for a real entire back end which contains the user application code (BE) and the embedded agents (EA). Embedded agents are linked with the back end and run on separate threads from the application code. Not all the back ends have embedded agents. A back end will receive configuration information during SCI_Initialize() which indicates if it should create embedded agents.

Transmitting Messages

The SCI_Bcast and SCI_Upload functions are the most important functions for transmitting messages. SCI_Bcast is used to send messages from front end to any of the back ends while SCI_Upload is used to send messages from a back end to the front end.

SCI provides a one-sided communication model for message transmission, that is, there is a message handler which is a call-back function registered when SCI_Initialize() is called. The call-back function is called automatically when a message is received. SCI has both interrupt mode and polling mode. The message handler is triggered by an incoming message in either mode. For interrupt mode, this handler is called in a separate SCI thread while for polling mode, it is called within the SCI_Poll() function. Typically, it is the main thread which is blocking on SCI_Poll() when waiting for a message.

Filtering

SCI provides a plug-in mechanism for messages filtering which is done in SCI agents (scia or EA, also FE). Only one filter can be used for downstream (FE->BEs) per agent while multiple filters can be cascaded for upstream (BEs->FE) per agent/embedded agent.

Groups

A group may contain one or multiple back end ids. Users can create groups and communicate with groups of back end through a set of group APIs. Any back end id is considered as a predefined group id consisting of only that back end. The SCI_GROUP_ALL is also a predefined group which contains all the back ends.

Internally in each agent/embedded agent/front end, each group has two kinds of information, the first is the back end ids belonging to this group, the second is the direct successors of this agent/embedded agent/front.

Launch modes

SCI implements two launch modes, internal launch and external launch. Internal launching means the SCI back ends are forked by scid directly when the front end calls SCI_Initialize(). The external launching mode is intended for when the back ends have to be launched by a third party job launcher other than scid, for example, the back end is a debug engine and it has to be launched by another job launcher such as POE/PMD. Then users can use this mode; just set the SCI_CLIENT_ID and SCI_JOB_KEY environment variables before the back end calls SCI_Initialize(), if the front end has the same job key and calls SCI_Initialize(), the back end can find its parent agent and connect back to the SCI tree hierarchy. Both the front end and back ends should export SCI_USE_EXTLAUNCHER=yes. The embedded agent mode can not used with this mode.

Working model

The figures below are two examples for typical programming models. The handler is triggered once a message arrives, which is the concept of one-sided communication in this design.

Working model 1

Working model 2