Notice: This Wiki is now read only and edits are no longer possible. Please see: https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/wikis/Wiki-shutdown-plan for the plan.
Difference between revisions of "SMILA/Documentation"
< SMILA
Line 1: | Line 1: | ||
− | + | == Basics == | |
− | * | + | * [[SMILA/Architecture_Overview|Architecture]] |
− | *[ | + | * [[SMILA/Default_configuration_workflow_overview|Overview on Default Configuration]] |
+ | * [[SMILA/Documentation/Data_Model_and_Serialization_Formats|Data Model, XML, JSON, BON]] | ||
+ | * [[SMILA/Documentation/Using_The_ReST_API|Using the ReST API]] | ||
+ | * [[SMILA/REST_API_Reference|REST API Reference]] | ||
− | + | == The SMILA Development Environment == | |
− | + | * [[SMILA/Development_Guidelines/Howto_set_up_dev_environment|Setup your Eclipse IDE for SMILA]] | |
− | + | * [[SMILA/Development_Guidelines/Howto_build_a_SMILA-Distribution|Build SMILA]] | |
− | + | * [[SMILA/Development_Guidelines/Create_a_bundle_(plug-in)|Creating new components]] | |
− | + | * [[SMILA/Development_Guidelines/Create_a_test_bundle_(plug-in)|Testing new components]] | |
− | + | * [[SMILA/Development_Guidelines/3rd_party_bundle|Adding Third Party libraries to SMILA]] | |
− | + | * [[SMILA/Development_Guidelines/Declarative_Services|Using OSGi Declarative Services]] | |
− | **[[SMILA/ | + | * Extending the build process: |
− | **[[SMILA/ | + | ** [[SMILA/Development_Guidelines/How_to_integrate_new_bundle_into_build_process|Adding a new component bundle to the build]] |
+ | ** [[SMILA/Development_Guidelines/How_to_integrate_test_bundle_into_build_process|Adding a new test bundle to the build]] | ||
− | *Connectivity | + | == Pipelines and Pipelets: Synchronous Workflows of Java components == |
+ | * [[SMILA/Documentation/Pipelets|What are Pipelines? What are Pipelets?]] | ||
+ | * [[SMILA/Documentation/BPEL_Workflow_Processor|Configuration and Creating BPEL Pipelines]] | ||
+ | ** [[SMILA/Development_Guidelines/How_to_filter_and_access_record_data_in_BPEL|Accessing SMILA records in BPEL activities]] | ||
+ | * [[SMILA/BPEL_Designer|Using the SMILA BPEL Designer]] | ||
+ | * ReST APIs | ||
+ | ** [[SMILA/Documentation/Processing/JSON_REST_API_for_pipelets|for showing pipelets]] | ||
+ | ** [[SMILA/Documentation/Processing/JSON_REST_API_for_BPEL_pipelines|for showing, manipulating and executing pipelines]] | ||
+ | * Basic Pipelets | ||
+ | ** [[SMILA/Documentation/Bundle org.eclipse.smila.processing.pipelets|Common pipelets: bundle org.eclipse.smila.processing.pipelets]] | ||
+ | ** [[SMILA/Documentation/Bundle org.eclipse.smila.processing.pipelets.xmlprocessing|Common XML pipelets: Bundle org.eclipse.smila.processing.pipelets.xmlprocessing]] | ||
+ | ** More special pipelets are provided by components described below. | ||
+ | * Developing Pipelets | ||
+ | ** [[SMILA/Development_Guidelines/How_to_write_a_Pipelet|How to write a pipelet]] | ||
+ | ** [[SMILA/Documentation/Usage_of_Blackboard_Service|BlackboardService]] | ||
+ | |||
+ | == Using SMILA for Search == | ||
+ | |||
+ | * [[SMILA/Documentation/Search|Search Processing and APIs]] | ||
+ | * [[SMILA/Documentation/Solr|Solr Integration: Configuration and Pipelets]] | ||
+ | |||
+ | == JobManager: Asynchronous Workflows == | ||
+ | * [[SMILA/Documentation/JobManager|Concepts: Jobs and Tasks]] | ||
+ | ** [[SMILA/Documentation/JobManagerFirstExample|A simple example]] | ||
+ | * Creating Workflows and Jobs | ||
+ | ** [[SMILA/Documentation/DataObjectTypesAndBuckets|Defining Buckets]] | ||
+ | ** [[SMILA/Documentation/WorkerAndWorkflows|Modeling Workflows]] | ||
+ | ** [[SMILA/Documentation/JobDefinitions|Creating Jobs]] | ||
+ | ** [[SMILA/Documentation/JobParameters|Evaluation of Job Parameters]] | ||
+ | * [[SMILA/Documentation/JobRuns|Running and Monitoring Jobs]] | ||
+ | * [[SMILA/Documentation/JobManagerConfiguration|JobManager Configuration]] | ||
+ | * [[SMILA/Documentation/TaskManager|TaskManager: Asynchronous scheduling of tasks]] | ||
+ | * Workers Reference | ||
+ | ** [[SMILA/Documentation/Bulkbuilder|Bulkbuilder]] | ||
+ | ** [[SMILA/Documentation/Worker/PipelineProcessorWorker|PipelineProcesor]] | ||
+ | ** [[SMILA/Documentation/Worker/PipeletProcessorWorker|PipeletProcessor]] | ||
+ | ** See [[SMILA/Manual#Importing|Importing]] below for more workers | ||
+ | * Developing Workers | ||
+ | ** [[SMILA/Documentation/WorkerManager|WorkerManager: Workers made easy]] | ||
+ | ** [[SMILA/Development_Guidelines/How_to_write_a_Worker|How to write a worker]] | ||
+ | ** [[SMILA/Documentation/TaskGenerators|Task Generators]] | ||
+ | |||
+ | == Importing == | ||
+ | * [[SMILA/Documentation/Importing/Concept|Concepts]] | ||
+ | * Importing Workers reference | ||
+ | **[[SMILA/Documentation/Importing/Crawler/File | FileCrawler and FileFetcher worker]] | ||
+ | **[[SMILA/Documentation/Importing/Crawler/Web | WebCrawler and WebFetcher worker]] | ||
+ | **[[SMILA/Documentation/Importing/DeltaCheck | DeltaCheck worker]] | ||
+ | **[[SMILA/Documentation/Importing/UpdatePusher | UpdatePusher worker]] | ||
+ | * Developing Importing Workers | ||
+ | ** [[SMILA/Documentation/Importing/VisitedLinks | VisitedLinks service]] | ||
+ | ** [[SMILA/Documentation/Importing/Crawler/Web#Internal_structure|Extending the WebCrawler worker]] | ||
+ | ** [[SMILA/Development_Guidelines/How to add a new Data Source to the importing framework|How to add a new '''Data Source''' to the SMILA importing framework]] | ||
+ | |||
+ | == The SMILA HTTP Server == | ||
+ | * [[SMILA/Documentation/JettyHttpServer|Configuration of Jetty]] | ||
+ | * [[SMILA/Documentation/JettyHttpServer#JSON_Handlers|Developing JSON ReST handlers for SMILA]] | ||
+ | |||
+ | == Common Services == | ||
+ | * [[SMILA/Project Concepts/Simple configuration handler|Configuration Helper]] | ||
+ | * [[SMILA/Documentation/Workspace Handling|Workspace Helper]] | ||
+ | * [[SMILA/Documentation/Bundle_org.eclipse.smila.clusterconfig|ClusterConfig service]] | ||
+ | ** [[SMILA/Documentation/Bundle_org.eclipse.smila.clusterconfig.simple|Simple implementation]] | ||
+ | * [[SMILA/Documentation/ObjectStore/Bundle_org.eclipse.smila.objectstore|ObjectStore]] | ||
+ | ** [[SMILA/Documentation/ObjectStore/Bundle_org.eclipse.smila.objectstore.filesystem|Filesystem Objectstore implementation]] | ||
+ | * [[SMILA/Documentation/Binary_Storage|BinaryStorage]] | ||
+ | * [[SMILA/Documentation/Security|Processing security information]] | ||
+ | ** [[SMILA/Documentation/LDAPSecurityResolver|Implementation for using LDAP]] | ||
+ | ** [[SMILA/Documentation/SecurityProcessingServices|Example Pipelets]] | ||
+ | * [[SMILA/Documentation/SesameOntologyManager|Ontology processing with Sesame: Configuration and Pipelets]] | ||
+ | * [[SMILA/Documentation/MimeTypeIdentifier|MimeTypeIdentifier]] | ||
+ | * [[SMILA/Documentation/PublishingJAXWSWebservices|Publishing Web Services]] | ||
+ | * [[SMILA/Documentation/General JPA Configuration in SMILA|General JPA Configuration in SMILA]] | ||
+ | |||
+ | == Deprecated Components == | ||
+ | * [[SMILA/Documentation/ConnectivityFramework|Connectivity Framework]] | ||
**[[SMILA/Documentation/ConnectivityManager|ConnectivityManager]] | **[[SMILA/Documentation/ConnectivityManager|ConnectivityManager]] | ||
**[[SMILA/Documentation/DeltaIndexingManager|DeltaIndexingManager]] | **[[SMILA/Documentation/DeltaIndexingManager|DeltaIndexingManager]] | ||
− | |||
***[[SMILA/Documentation/CrawlerController|CrawlerController]] | ***[[SMILA/Documentation/CrawlerController|CrawlerController]] | ||
***[[SMILA/Documentation/Crawler|Crawler]] | ***[[SMILA/Documentation/Crawler|Crawler]] | ||
Line 28: | Line 106: | ||
****[[SMILA/Documentation/JobFile Agent|JobFile Agent]] | ****[[SMILA/Documentation/JobFile Agent|JobFile Agent]] | ||
***[[SMILA/Documentation/CompoundManagement|CompoundManagement]] | ***[[SMILA/Documentation/CompoundManagement|CompoundManagement]] | ||
− | + | ** Developing | |
− | * | + | *** [[SMILA/Development_Guidelines/How_to_implement_a_Crawler|How to implement a crawler]] |
− | * | + | *** [[SMILA/Development_Guidelines/How_to_implement_an_agent|How to implement an agent]] |
− | * | + | ** [[SMILA/Documentation/Management|JMX Management]] |
− | **[[SMILA/ | + | *** [[SMILA/Documentation/Management#JMX_Client|JMX Clients]] |
− | * | + | ** [[SMILA/Documentation/Record_Storage|RecordStorage]] |
− | **[[SMILA/ | + | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | *[[SMILA/Documentation/Management|Management]] | + | |
− | + | ||
− | * | + | |
− | **[[SMILA/Documentation/Management#JMX_Client|JMX | + | |
− | **[[SMILA/Documentation/ | + | |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
[[Category:SMILA]] | [[Category:SMILA]] |
Revision as of 09:21, 24 January 2012
Contents
Basics
- Architecture
- Overview on Default Configuration
- Data Model, XML, JSON, BON
- Using the ReST API
- REST API Reference
The SMILA Development Environment
- Setup your Eclipse IDE for SMILA
- Build SMILA
- Creating new components
- Testing new components
- Adding Third Party libraries to SMILA
- Using OSGi Declarative Services
- Extending the build process:
Pipelines and Pipelets: Synchronous Workflows of Java components
- What are Pipelines? What are Pipelets?
- Configuration and Creating BPEL Pipelines
- Using the SMILA BPEL Designer
- ReST APIs
- Basic Pipelets
- Common pipelets: bundle org.eclipse.smila.processing.pipelets
- Common XML pipelets: Bundle org.eclipse.smila.processing.pipelets.xmlprocessing
- More special pipelets are provided by components described below.
- Developing Pipelets
Using SMILA for Search
JobManager: Asynchronous Workflows
- Concepts: Jobs and Tasks
- Creating Workflows and Jobs
- Running and Monitoring Jobs
- JobManager Configuration
- TaskManager: Asynchronous scheduling of tasks
- Workers Reference
- Bulkbuilder
- PipelineProcesor
- PipeletProcessor
- See Importing below for more workers
- Developing Workers
Importing
- Concepts
- Importing Workers reference
- Developing Importing Workers
The SMILA HTTP Server
Common Services
- Configuration Helper
- Workspace Helper
- ClusterConfig service
- ObjectStore
- BinaryStorage
- Processing security information
- Ontology processing with Sesame: Configuration and Pipelets
- MimeTypeIdentifier
- Publishing Web Services
- General JPA Configuration in SMILA