JDBC Index Stage

The JDBC Index Stage is used to connect to a database, lookup one or more values, and then inject them into the context. The properties for setting stage are identical to the JDBC Query Stage, except for the “rows” property, which defaults to -1 for the index stage (which returns all rows in the database), and defaults to 10 returned rows for the query stage.

You must first upload a JDBC Driver to Fusion.

Upload a JDBC Driver to Fusion

The JDBC V2 connector is supported, and fetches documents from a relational database via SQL queries. Under the hood, this connector implements the Solr DataImportHandler (DIH) plugin.

Fusion stores JDBC drivers in the blob store. You can upload a driver using the Fusion UI or the Blob Store API.

How to upload a JDBC driver using the Fusion UI

1. In the Fusion UI, navigate to System > Blobs.

2. Click Add.

3. Select JDBC Driver.
   The “New ‘JDBC Driver’ Upload” panel appears.

   

4. Click Choose File and select the .jar file from your file system.

   

5. Click Upload.
   The new driver’s blob manifest appears.

   

From this screen you can also delete or replace the driver.

How to install a JDBC driver using the API

1. Upload the JAR file to Fusion’s blob store using the /blobs/{id} endpoint.

   Specify an arbitrary blob ID, and a resourceType value of plugin:connector, as in this example:

   curl -u USERNAME:PASSWORD -H "content-type:application/java-archive" -H "content-length:707261" -X PUT --data-binary @postgresql-42.0.0.jar http://localhost:8764/api/blobs/mydriver?resourceType=driver:jdbc
   

   Success response:

   {
     "name" : "mydriver",
     "contentType" : "application/java-archive",
     "size" : 707261,
     "modifiedTime" : "2017-06-09T19:00:48.919Z",
     "version" : 0,
     "md5" : "c67163ca764bfe632f28229c142131b5",
     "metadata" : {
       "subtype" : "driver:jdbc",
       "drivers" : "org.postgresql.Driver",
       "resourceType" : "driver:jdbc"
     }
   }
   

   Fusion automatically publishes the event to the cluster, and the listeners perform the driver installation process on each node.

   If the blob ID is identical to an existing one, the old driver will be uninstalled and the new driver will installed in its place. To get the list of existing blob IDs, run: curl -u USERNAME:PASSWORD https://FUSION_HOST:FUSION_PORT/api/blobs

2. To verify the uploaded driver, run:

   curl -u USERNAME:PASSWORD https://FUSION_HOST:FUSION_PORT/api/blobs/BLOB_ID/manifest
   

   Where the BLOB_ID is the name specified during upload, such as “mydriver” above. A success response looks like this:

   {
     "name" : "mydriver",
     "contentType" : "application/java-archive",
     "size" : 707261,
     "modifiedTime" : "2017-06-09T19:05:17.897Z",
     "version" : 1569755095787110400,
     "md5" : "c67163ca764bfe632f28229c142131b5",
     "metadata" : {
       "subtype" : "driver:jdbc",
       "drivers" : "org.postgresql.Driver",
       "resourceType" : "driver:jdbc"
     }
   }
   

Example

An example of a JDBC Index Stage setup

Upload stage config via POST to Fusion REST API endpoint api/index-stages/instances

curl -u USERNAME:PASSWORD -X POST -H "Content-Type: application/json" -d '{"id": "jdbc-index-test","type":"jdbc-index-lookup","driver":"postgresql-9.3-1101-jdbc4.jar","connection":"jdbc:postgresql:database","username":"user","password":"password1","preparedStatement":"select ID as id from DATABASE;"}' http://FUSION_HOST:FUSION_PORT/api/index-stages/instances

Response

{
  "type" : "jdbc-index-lookup",
  "id" : "jdbc-index-test",
  "driver" : "postgresql-9.3-1101-jdbc4.jar",
  "connection" : "jdbc:postgresql:database",
  "username" : "user",
  "password" : "password1",
  "preparedStatement" : "select ID as id from DATABASE;",
  "fetchSize" : -1,
  "join" : true,
  "rows" : -1,
  "skip" : false,
  "label" : "jdbc-index-lookup",
  "type" : "jdbc-index-lookup"
}

Configuration

When entering configuration values in the UI, use unescaped characters, such as \t for the tab character. When entering configuration values in the API, use escaped characters, such as \\t for the tab character.