Revision as of 14:30, 30 March 2011

Flex Columns Extension

This feature allows mappings to be added to preexisting (unmapped) columns in a table.

The Schema is designed to include preallocated columns that can be used to map additional data. In this example, Customer table might look like this:

• CUSTOMER
  • INTEGER ID
  • VARCHAR NAME
  • VARCHAR FLEX_COL1
  • VARCHAR FLEX_COL2
  • VARCHAR FLEX_CO31

The columns starting with the String "FLEX_COL" are flexible and may have mappings that point to them added at any time. For instance at runtime, a user may add a mapping called "company" that maps to FLEX_COL1.

Requirements

• Users MUST be able to add extensions at Runtime
• Extensions MUST be persistent. (i.e. Extensions must continue to exist if an application goes down and comes back up)
• Extensions MUST be shared amoung all EntityManagers configured to use them
• Extensions MUST be compatible with our multi-tenant features. (i.e. Extensions must be able to make use of our Multi-tenant features to be definable on a tenant by tenant basis)
• Extensions MUST have DDL Generation support
• BasicMappings MUST be supported
• Extensibility must be configurable using tradition JPA means (annotations, eclispelink-orm.xml)
• It MUST be possible to use JPA Queries to query based on the extensions
• Extensions SHOULD be supported through the JPA metamodel
• OneToOneMappings SHOULD be supported

Configuration

Metadata

Extensions will be supported through a user-defined map that uses the extension name as the key and the value of the extension as the map value. The user will be required specify that map on their domain class. Extensions will be updated by making changes to that map.

Annotations and xml will be used to set the map as holding the extensions. A validation exception will be thrown if more than one map is configured as holding extensions on a given class.

When defining that Map, the following information is supplied

• numberOfColunms (default 10) - the number of flexible columns available
• columnNamePrefix (default "FLEX_") - the prefix for the column name. When EclipseLink autogenerates the columns a number will be provided as a suffix to complete the column name. (i.e. the first column will be called FLEX_1 and the second column will be called FLEX_2)
• defaultColumnType (default VARCHAR(255) - the default type of the columns
• extensionTableName (default FLEX_DEF) - the name of the table to store metadata about FLEX columns in. This table can be shared by extensions for multiple entities
• createNonExistingColumns - Hook for ALTER TABLE Functionality. Link to feature doc will be provided later
• columns (default is an emptyList) - Note: This functionality will be implemented in our 2nd iteration of development. This is an override column that allows you to override the names and types of certain columns and contains a list of @FlexColumn annotations.

@FlexColumn contains: (Note: as above FlexColumn will be implemented in our 2nd iteration of development)

• index - required - the index of the column to update
• name - an override for the default column name provided
• type - an override for the default type provided

API

ClassDescriptor will hold an ExtensionManager.

ExtensionManager API

• addExtension(extensionName, extensionJavaType, extensionField) - add an extension using a specific database field
• addExtension(extensionName, extensionJavaType) - add an extension and allow EclipseLink to choose a database field
• getExtensionFields() get all extension fields
• getAvailableExtensionFields() return all of the unused extension fields
• removeExtension(name)

JpaHelper API

• addExtension(emf, entityName, extensionName, extensionJavaType, extensionField) - call the addExtension method for the descriptor for entityName with extensionName, extensionJavaType and extensionField
• addExtension(emf, entityName, extensionName, extensionJavaType) - call the addExtension method for the descriptor for entityName with extensionName, and extensionJavaType
• getExtensionFields(emf, entityName) - call getExtensionFields() on the descriptor for entityName
• getAvailableExtensionFields(emf, entityName) - call getAvailbleExtensionFields() on the descriptor for entityName
• removeExtension(EntityManagerFactory emf, String entityName, String extensionName) - call the removeExtension method for the descriptor for entityName with extensionName

Example

  @FlexExtensions(columnNamePrefix="FLEX",
      numberOfColumns=10,
      defaultColumnType="VARCHAR(30)"     
      createNonExistingColumns=false, 
      extensionTableName="EXTS")
  private Map<String, Object> extensions = new HashMap<String, Object>();
 
  public <T> T get(String name) {
      return (T) getExtensions().get(name);
  }
 
  public Object set(String name, String value) {
      return getExtensions().put(name, value);
  }

Design

UNDER CONSTRUCTION

FlexExtensionManager

Each ClassDescriptor may have an ExtensionManager. For extensible types, the ExtensionManager will be non-null and will be set either by annotation, xml, or explicitly on the descriptor. Different types of ExtensionManager are possible. The flex column behavior is managed by a FlexExtensionManager.

FlexExtensionManager holds the following state:

• extensionsAccessor - an Accessor that gives it access to the Map on the domain class that holds the extensions. e.g. the Map annotated as @FlexExtensions
• availableFields - the list of DatabaseField that it believes can be used for new extensions
• usedFields - the list of DatabaseField that it believes is used for existing extensions
• shouldCreateNonExistingColumns - a boolean used to enable the additional feature that will allow ALTER schema language to create new columns (more to come)

Extension Data

Extension data will be persisted to the database. Each extension will be represented by an ExtensionProperty holding the following data:

• String - entityType - the name of the Entity it represents
• DatabaseField - field - a DatabaseField representing the column used by this extension
• Class - type - the target class of the mapping
• String - name - the name of the extension
• mapping - a reference to the mapping for this extension

An project containing an Extensible type will have a descriptor added to it for ExtensionProperty. Any time an Extension is added, its ExtensionProperty will be persisted. The table will contain the following information:

• ENTITY_TYPE (PK) - Directly mapped from entityType
• COLUMN_NAME (PK) - Name of the field stored in field
• TYPE - Class name of the class represented by type
• NAME - Directly mapped from name

At descriptor initialization time, a ReadAllQuery will be issued for all ExtensionProperties with an ENTITY_TYPE. These will be stored by the ExtensionManager and used to build the appropritate mappings.

Adding Extensions

Extensions are added through a call to extensionManager.addExtension(name, type). When addExtension is called, the following will occur:

1. a check will be made to ensure the extension does not already exist in the descriptor and an exception will be thrown if it does
2. the list of available fields will be checked for a field that can represent the type that is passed in
3. If no field is available either the Column Creation feature will be used, or an exception will be thrown.
4. the availableFields and usedFields lists will be updated
5. An instance of ExtensionProperty will be created and persisted, if persisting fails for any reason, an exception will be thrown and the availableFields and usedFields lists will be reset
6. A mapping will be created to represent the ExtensionProperty and it will be added to the descriptor

Mapping Types

Extensions may only be the following types of mappings

• Basic
• OneToOne (single foreign key)

Basic Mappings

The initial iteration will allow Basic mappings. Basic mappings are fairly easy to implement as they are a simple coorespondance between a field and value in the map. A basic mapping will be constructed with the appropriate data and added to the descriptor

OneToOne Mappings

More to come...

Extension Propogation

Describe how extensions are shared between clients here

Discussion: What are the requirements

• Does this have to be a PUSH-style notification, or could it be at EntityManager or EntityManagerFactory creation time?

DDL Generation

DDL generation will allow the creation of the extension table and also include any known extension columns to domain tables

Column Creation

See <URL for Column Creation doc here>

Appendices

Apendix 1: Annotations

@Target({TYPE}) 
@Retention(RUNTIME)
public @interface FlexExtensions {
    /**
     * (Optional) Specify number of columns.  This is used to automatically generate a default set of columns. 
     * Unnecessary if columns is specified
     */
    int numberOfColumns default 10;
 
    /**
     * The string to prefix to the name of automatically generated columns.
     * e.g. the first flex column could be "FLEX_COL1" and the second, "FLEX_COL2" etc
     */
    String columnNamePrefix default "FLEX_COL";
 
    /**
     * The string to prefix to the name of automatically generated columns.
     * e.g. the first flex column could be "FLEX_COL1" and the second, "FLEX_COL2" etc
     */
    String defaultColumnType default VARCHAR(255);
 
    /**
     * When set to true this will activate a feature that executes ALTER table statements to 
     * make the required columns available if they do not exist
     */
    boolean createNonExistingColumns default false;
 
    /**
     *  The name of the table that holds the extensions to use.  This table can be shared.
     */
    String extensionTableName default null;
 
    /**
     * (Optional) Define column overrides
     */
    FlexColumn[] columns
}
 
@Target({TYPE}) 
@Retention(RUNTIME)
public @interface FlexColumn {
 
    /**
     * Mandatory - the index of the column to override.  Between 1 and numberOfColumns.
     * Exception will be thrown for numbers < 1 or > numberOfColumns
     */
    int index;
 
    /**
     * Column name - used to override default column names
     */
    String name;
 
    /**
     * Column type - used to override default column type
     */
    String type default VARCHAR(255)
}

Appendix 2: eclipselink-orm

  <xsd:complexType name="attributes">
    <xsd:annotation>
      <xsd:documentation>
 
        This element contains the entity field or property mappings.
        It may be sparsely populated to include only a subset of the
        fields or properties. If metadata-complete for the entity is true
        then the remainder of the attributes will be defaulted according
        to the default rules.
 
      </xsd:documentation>
    </xsd:annotation>
    <xsd:sequence>
    ...
    <xsd:element name="flex-extensions" type="orm:flex-extensions" minOccurs="0"/>
 
<!-- **************************************************** -->
 
<xsd:complexType name="flex-extensions">
  <xsd:annotation>
    <xsd:documentation>
      ...
    </xsd:documentation>
  </xsd:annotation>
  <xsd:sequence>
    <xsd:attribute name="number-of-columns" type="xsd:integer"/>
    <xsd:element name="column-name-prefix" type="xsd:string"/>
    <xsd:attribute name="default-column-type" type="xsd:string"/>
    <xsd:attribute name="create-non-existing-columns" type="xsd:boolean"/>
    <xsd:attribute name="extension-table-name" type="xsd:string"/>
    <xsd:element name="flex-columns" type="eclipselink-orm:flex-column" minOccurs="0" maxOccurs="unbounded"/>
  </xsd:sequence>
</xsd:complexType>
 
<xsd:complexType name="flex-column">
  <xsd:annotation>
    <xsd:documentation>
      ...
    </xsd:documentation>
  </xsd:annotation>
  <xsd:sequence>
    <xsd:attribute name="index" type="xsd:integer"/>
    <xsd:attribute name="name" type="xsd:string"/>
    <xsd:attribute name="type" type="xsd:string"/>
  </xsd:sequence>
</xsd:complexType>