Striim

Welcome to the Striim Help Center and Community Site

3.10.3 Configuring Striim

Follow

This section describes how to configure a Striim cluster.

Installing third-party drivers

This section describes installation of the third-party drivers required to use some adapters and other Striim features.

Installing the HP NonStop JDBC driver for SQL/MX

To read from or write to HP NonStop SQL/MX tables using Database Reader or Database Writer, the HP NonStop JDBC driver must be present in the present in Striim's Java classpath.

Caution

Do not install drivers for multiple versions of SQL/MX on the same Striim server. If you need to write to multiple versions of SQL/MX, install their drivers on different Striim servers and run each version's applications on the appropriate Striim server(s).

  1. Follow the instructions in the "Installing and Verifying the Type 4 Driver" section of HPE NonStop JDBC Type 4 Driver Programmer's Reference for SQL/MX for the SQL/MX version you are running to copy the driver .tar file from the HP NonStop system with the tables that will be read to a client workstation and untar it. Do not install the driver.

  2. Copy the t4sqlmx.jar file from the untarred directory to striim/lib or Agent/lib.

  3. Stop and restart Striim (see Starting and stopping Striim).

Installing the MariaDB JDBC driver

To use MariaDB Reader, its JDBC driver must be present in the Java classpath for the Striim server. (When reading from MariaDB using Database Reader, Incremental Batch Reader, or MySQL Reader, Striim uses the MySQL JDBC driver.)

  1. Download mariadb-java-client-2.4.3.jar from http://downloads.mariadb.com/Connectors/java/connector-java-2.4.3.

  2. Copy that file to striim/lib or Agent/lib.

  3. Stop and restart Striim (see Starting and stopping Striim).

Installing the MemSQL JDBC driver

MemSQL uses MySQL's JDBC driver. See Installing the MySQL JDBC driver.

Installing the Microsoft JDBC driver

To read from or write to Microsoft SQL Server, Azure SQL Database, or Azure SQL Data Warehouse, the Microsoft JDBC Driver for SQL Server must be present in Striim's Java classpath.

mssql-jdbc-7.2.2.jre8.jar, the Microsoft JDBC driver for SQL Server 2012 and later, is bundled with the Striim server

That driver is not bundled with the Forwarding Agent, so to read from those sources, you must install one of the drivers.

Caution

Do not install both versions of the driver in the same Striim server or Forwarding Agent.

For SQL Server 2008:

  1. On Striim servers, delete striim/lib/mssql-jdbc-7.2.2.jre8.jar.

  2. Download the Microsoft JDBC Driver 6.0 for SQL Server .gz package from https://www.microsoft.com/en-us/download/details.aspx?id=11774 and extract it.

  3. Copy enu/jre8/sqljdbc42.jar to striim/lib. or agent/lib.

  4. Stop and restart Striim (see Starting and stopping Striim).

For all other versions, for the Forwarding Agent only:

  1. Download the Microsoft JDBC Driver 7.2 for SQL Server .gz package from https://www.microsoft.com/en-us/download/details.aspx?id=57782 and extract it.

  2. Copy enu/mssql-jdbc-7.2.2.jre8.jar to agent/lib.

  3. Stop and restart Striim (see Starting and stopping Striim).

Installing the MySQL JDBC driver

To use MySQLReader, to use DatabaseReader or DatabaseWriter with MySQL or MariaDB, or to persist a WActionStore to MySQL, the MySQL Connector/J JDBC driver must be present in the Java classpath for the Striim server. (To write to MariaDB Galera Cluster with DatabaseWriter, see Installing the MariaDB JDBC driver.)

  1. Download the Connector/J 5.1.46 package from https://dev.mysql.com/downloads/connector/j/5.1.html and extract it.

  2. Copy mysql-connector-java-5.1.46.jar to striim/lib or Agent/lib.

  3. Stop and restart Striim (see Starting and stopping Striim).

Installing the Oracle Instant Client

To use OracleReader with XStream Out you must perform the following steps for each Striim server or Forwarding Agent that will run the OracleReader source. The Striim servers and Forwarding Agent hosts must be running Linux.

  1. Download the Oracle Instant Client .rpm file for your version of Oracle (11.2.0.2, 11.2.0.3, or 11.2.0.4) from http://www.oracle.com/technetwork/topics/linuxx86-64soft-092277.html and install it with rpm -i -v <file name>. The instructions below assume that the installation directory is /usr/lib/oracle/instantclient_11_2. If it is in a different location, adjust the environment variables accordingly.

  2. Add /usr/lib/oracle/instantclient_11_2/xstreams.jar:/usr/lib/oracle/instantclient_11_2/ojdbc6.jar to the CLASSPATH environment variable. In Centos, if Striim is owned by root, you may do this by editing /root/.bash_profile. For example, if the PATH portion of that file is:

    PATH=$PATH:$HOME/bin
    export PATH
    

    change that to:

    PATH=$PATH:$HOME/bin:/usr/lib/oracle/11.2/client64/lib
    export PATH
    
  3. Add /usr/lib/oracle/instantclient_11_2/ to the LD_LIBRARY_PATH environment variable.

  4. Add /usr/lib/oracle/instantclient_11_2/ to the PATH environment variable.

  5. Add the IP address and name of the Oracle server(s) to which you will connect to the hosts file.

Installing the Oracle JDBC driver

The Oracle JDBC drier is bundled with the Striim server, but must be installed on each Forwarding Agent that will run Oracle Reader with LogMiner.

Perform the following steps on each Striim server and Forwarding Agent that will run an OracleReader source using LogMiner:

  1. Download ojdbc8.jar from oracle.com.

  2. Save that file in striim/lib or agent/lib.

  3. Stop and restart Striim (see Starting and stopping Striim).

Installing the PostgreSQL JDBC driver

Starting with Striim 3.9.0, this driver is included with the Striim server.

Perform the following steps on each Forwarding Agent that will read from PostgreSQL.

  1. Download the PostgreSQL JDBC 4.2 driver from jdbc.postgresql.org/download.html.

  2. Copy it to agent/lib.

  3. Restart the agent.

Installing the Redshift JDBC driver

To write to Redshift using RedshiftWriter, its JDBC 4.1 driver must be present in the Java classpath for the Striim server.

Warning

You can not reliably read from or write to PostgreSQL or use it to host Striim's metadata repository when the Redshift JDBC driver is installed.

  1. Download the JDBC 4.1 driver from http://docs.aws.amazon.com/redshift/latest/mgmt/configure-jdbc-connection.html#download-jdbc-driver.

  2. Copy the .jar file to striim/lib.

  3. Stop and restart Striim (see Starting and stopping Striim).

Installing the SAP HANA JDBC driver

To write to SAP HANA using DatabaseWriter, its JDBC driver must be present in the Java classpath for the Striim server.

  1. Download the JDBC 2.4.62 package from store.sap.com and extract it.

  2. Copy ngdbc-2.4.62.jar file to striim/lib.

  3. Stop and restart Striim (see Starting and stopping Striim).

Installing the Snowflake JDBC driver

To write to Snowflake using SnowflakeWriter, its JDBC driver must be present in the Java classpath for the Striim server.

  1. Download the appropriate Snowflake JDBC driver from https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc.

  2. Copy the downloaded file to striim/lib.

  3. Restart the Striim server or agent.

Installing the Teradata JDBC driver

To use DatabaseReader or IncrementalBatchReader with Teradata, the Teradata JDBC Driver must be present in the Java classpath for the Striim server. Perform the following steps on each Striim server or Forwarding Agent that will read from Teradata.

  1. Download the Teradata JDBC .tgz or .zip package from http://downloads.teradata.com/download/connectivity/jdbc-driver and extract it.

  2. Copy tdgssconfig.jar and terajdbc4.jar to striim/lib. or agent/lib.

  3. Stop and restart Striim (see Starting and stopping Striim).

Changing the application start timeout

By default, when you start an application, Striim will wait five minutes for all components and flows to start. If after that time not all components and flows have started, start will fail with an error such as "failed to verify flow start."

To change this timeout, edit startUp.properties (and, if the application has a source running in a Forwarding Agent, agent.properties) and set the value of striim.cluster.maxWaitTimeForAppVerifyInDeploying to the number of seconds before timeout. Then restart Striim and any relevant Forwarding Agent.

Changing the Derby password

In a production environment, we strongly recommend changing the Derby password from its default. The JAVA_HOME environment variable must be set for the ./changeDerbyPassword.sh script to work.

  • Stop Striim and Derby (see Starting and stopping Striim).

  • Open a terminal and change to the striim directory.

  • Change the password in Derby:

    • In CentOS or Ubuntu, enter sudo su - striim bin/changeDerbyPassword.sh

    • In OS X, enter bin/changeDerbyPassword.sh.

    • In Windows, enter bin\changeDerbyPassword.

  • When prompted, enter the username waction, the current password (in a new installation, the password is w@ct10n), and the new password.

  • Change the password in Striim's keystore:

    • In CentOS or Ubuntu, enter sudo su - striim bin/sksConfig.sh -p

    • In OS X, enter bin/sksConfig.sh -p

    • In Windows, enter bin\sksConfig -p

  • Restart Derby and Striim (see Starting and stopping Striim).

Enabling file lineage

By default, file lineage is disabled.

To enable it for adapters other than OracleReader (see File lineage in readers and writers):

  • On the server running the source, edit striim/conf/startUp.properties, set TrackFLM=true, and restart the server as described in Starting and stopping Striim.

  • On the Forwarding Agent running the source, edit agent/conf/agent.conf, set striim.cluster.trackFileLineageMetadata=true, and restart the agent as described in Starting and stopping Striim.

To enable it for OracleReader (see File lineage in Oracle):

  • On the server running the source, edit striim/conf/startUp.properties, set TrackOLM=true, and restart the server as described in Starting and stopping Striim.

  • On the Forwarding Agent running the source, edit agent/conf/agent.conf, set striim.cluster.trackOracleLineageMetadata=true, and restart the agent as described in Starting and stopping Striim.

Configuring HTTP and HTTPS

With its default settings, Striim uses HTTP and HTTPS as follows:

  • The console discover the server using HTTP over port 9080, then connects using HTTPS over port 9081.

  • The Forwarding Agent authenticates using HTTPS over port 9081. After that, it joins the cluster using Hazelcast.

  • The web UI connects to the server using HTTP over port 9080. Alternatively, you may configure it to use HTTPS (Enabling HTTPS).

Changing the HTTP port

To change the HTTP port, on every server in the cluster, edit striim\conf\startUp.properties, set the HttpPort property value to the new port number, save the file, and restart Striim.

When the HTTP port is not 9080:

  • When starting the console, include the -t <port> switch to specify the HTTPS port.

  • Tell web UI users to use the new port.

Disabling HTTP

To disable HTTP, on every server in the cluster, edit ./striim/conf/startUp.properties, set HttpEnabled=false, and restart Striim.

When HTTP is disabled:

Changing the HTTPS port

To change the HTTPS port, on every server in the cluster, edit striim\conf\startUp.properties, set the HttpsPort property value to the new port number, save the file, and restart Striim.

When the HTTPS port is not 9081:

  • When starting the console, include the -T <port> switch to specify the HTTPS port.

  • In the Forwarding Agent's agent.conf configuration file, set the striim.node.httpsPort property value to the new port number and restart the agent.

  • If the web UI is using HTTPS, tell web UI users to use the new port.

Disabling HTTPS

To disable HTTPS (not recommended), on every server in the cluster, edit ./striim/conf/startUp.properties, set HttpsEnabled=false, and restart Striim.

When HTTPS is disabled:

  • When starting the console, include the -H false switch.

  • In the Forwarding Agent's agent.conf configuration file, set HttpsEnabled=False, set striim.node.servernode.address to the IP address of the server, and restart the agent.

Enabling HTTPS

To support HTTPS for the web UI, Striim needs an SSL certificate.

  1. If you already have a .pkcs12 file, skip to the next step.

    If you have .key and .crt files, use the following command to generate a .pkcs12 file (replace myfile with your certificate name):

    openssl pkcs12 -inkey myfile.key -in myfile.crt -export -out myfile.pkcs12
  2. Use the following command to generate a keystore containing the certificate from the .pkcs12 file (replace myfile with your certificate name)::

    keytool -importkeystore -srckeystore myfile.pkcs12 -srcstoretype PKCS12 -destkeystore myfile.jks

Alternatively, see Generate Keys to create a new self-signed certificate. In some browsers, these may trigger warnings about untrusted certificates.

Once the certificate is in Striim's environment, configure the following options in startUp.properties:

property

comments

HttpsKeystorePath

path to the keystore created by keytool, for example, /opt/Striim/myfile.jks

HttpsKeystorePassword

the keystore password (storepass value) specified when running keytool

HttpsKeystoreManagerPassword

the key password (keypass value) specified when running keytool or, if it was not specified, the storepass value

Setting the log levels

Log levels for striim.server.log, striim.command.log, and (for DEB and RPM installations) striim.command.log are set in Striim/conf/log4j.server.properties. When installed from .tgz, the default log level is info, when installed from DEB or RPM, the default is warn. From most to least verbose, the supported levels are alltracedebuginfowarnerror, and fatal.

Note

Debug-level UserCommandLogger messages, since they are often necessary for troubleshooting, are always written to striim.server.log, regardless of the log level setting.

Log levels for the Forwarding Agent are set in Agent/conf/log4j.agent.properties. The default log level is trace.

Changing log file retention settings

Default retention for striim.server.log and striim.agent.log is eight 1 GB files. You may adjust this by changing the log4j.appender.logfile.MaxBackupIndex or log4j.appender.logfile.MaxFileSize property values in striim/conf/log4j.server.properties or agent/conf/log4j.agent.properties.

Changing the amount of memory available to a Striim server

By default, a Striim server uses a maximum of 4GB.

To change that, edit striim/conf/startUp.properties and change the value of MEM_MAX. To increase maximum memory to, for example, 8GB, change the value to 8192m:

MEM_MAX=8192m

Then restart the server as described in Starting and stopping Striim.

Changing metadata repository connection retry settings

By default, when Striim tries to write to the metadata repository and it is temporarily unavailable due to network problems or other issues, Striim will wait ten seconds and try again. If that attempt fails, it will wait another ten seconds and try a third time. If the third attempt fails, the server will crash.

Similarly, when an application for which recovery is enabled tries to persist WActionStore data to a MySQL or Oracle database that is unavailable, it will try again twice before crashing. (When an application for which recovery is not enabled tries to persist WActionStore data to an unavailable database, it will crash immediately.)

To change the number of attempts or the time between them, change the following property values in startUp.properties, then restart Striim.

DBConnectionMaxRetries=3
DBConnectionRetryInterval=10

Changing how long monitor report data is maintained

By default, the historical data for monitor reports (see Using monitor reports) is retained for 24 hours. You may revise this by setting the MonitorPersistenceRetention property in startUp.properties to some other number of hours. For example, to retain the data for one week:

MonitorPersistenceRetention=168

Enabling monitoring via JMX

By default, monitoring via JMX is disabled. To enable it, edit striim/conf/startUp.properties, set EnableJmx=true, and restart the server as described in Starting and stopping Striim.

Enabling SSL for LDAP

To enable LDAP over SSL, install a certificate for the LDAP server in the Java runtime environment used by Striim using a command such as:

sudo keytool -import -alias <LDAP server IP address>
  -keystore $JAVA_HOME/jre/lib/security/cacerts -file public.crt

Supporting Active Directory authentication for Azure

When reading from or writing to Azure SQL Database or writing to Azure SQL Data Warehouse, you may use Active Directory authentication. This is not supported in sources running in a Forwarding Agent.

Caution

Updating .jar files in striim/lib may cause problems with other Striim functions that use them. If you encounter problems, delete the files added in step 3, restore the files in striim/lib.backup, and restart Striim.

  1. Follow the instructions in Installing the Microsoft JDBC driver.

  2. Move the following files from striim/lib to striim/lib.backup:

    • adal4j-1.0.0.jar

    • asm-all-5.0.3.jar

    • commons-codec-1.10.jar

    • commons-lang-2.6.jar

    • gson-2.2.4.jar

    • jcip-annotations-1.0.jar

    • json-smart-1.1.1.jar

    • lang-tag-1.4.jar

    • nimbus-jose-jwt-3.1.2.jar

    • oauth2-oidc-sdk-4.5.jar

  3. Download the following from mvnrepository.com and copy them to striim/lib:

    • accessors-smart-1.2.jar

    • adal4j-1.6.4.jar

    • asm-5.0.4.jar

    • commons-codec-1.11.jar

    • commons-lang3-3.5.jar

    • gson-2.8.0.jar

    • javax.mail-1.6.1.jar

    • jcip-annotations-1.0-1.jar

    • json-smart-2.3.jar

    • lang-tag-1.4.4.jar

    • nimbus-jose-jwt-8.2.1.jar

    • oauth2-oidc-sdk-6.5.jar

  4. Restart Striim.

.

To use Active Directory authentication, in the source or target adapter properties, for Username enter the fully qualified Active Directory user name, for Password enter its password, and use the following syntax for the Connection URL:

jdbc:sqlserver://<SQL Server IP address>:<port>;authentication=ActiveDirectoryPassword;
hostNameInCertificate=*.database.windows.net;

See Connecting using Azure Active Directory authentication for more information. Striim does not support the ActiveDirectoryIntegrated or ActiveDirectoryMSI properties.

Supporting Active Directory authentication for SQL Server

When Striim or a Forwarding Agent is running in Windows, you may use Active Directory authentication.

  1. Follow the instructions in Installing the Microsoft JDBC driver.

  2. Copy sqljdbc_4.2\enu\auth\x64\sqljdbc_auth.dll from the extracted JDBC driver package to the c:\windows\system32\ directory.

  3. Restart Striim or the Forwarding Agent.

In the source or target adapter properties, enter any values for Username and Password (they will be ignored), and use the following syntax for the Connection URL:

jdbc:sqlserver://<SQL Server IP address>:<port>;integratedSecurity=true

Striim or the Forwarding Agent will use the Active Directory credentials for the user account of its process to authenticate with SQL Server.

Using TCP/IP instead of multicast UDP

By default, as discussed in System requirements, a Striim cluster uses multicast UDP to discover and add servers. Alternatively, you may configure a cluster manually using TCP/IP. This will have no impact on performance.

To do that, on each server in the cluster, edit striim/conf/startUp.properties and set the following options:

IsTcpIpCluster=true
ServerNodeAddress=<node list>

The ServerNodeAddress value is a comma-separated list of the IP addresses of all servers in the cluster, for example, ServerNodeAddress=192.0.0.1,192.0.0.2.

Warning

The ServerNodeAddress setting must be the same on all servers in the cluster.

You must also open port 5701 for TCP on each server.

Then restart the servers, as described in Starting and stopping Striim.

Changing the web UI theme

By default, some pages in the Striim web UI have dark backgrounds and some have light backgrounds. To make all pages have consistent backgrounds, on every server that hosts the web UI, edit striim/webui/theme.conf and change the Theme property to light . Then restart Striim.

Note

Changing the theme may require changes to dashboards. For example, labels created with the Value visualization that have white text and a transparent background are readable against the dark gray background of the default theme, but the text color or background must be changed to make them readable against the white background of the light theme.

Note

Users may have to clear their browser caches for a change to the theme to take effect.

Changing the web UI ports

By default, the Striim web UI uses port 9080 for http and port 9081 for https. If you wish to set the ports manually, edit striim/conf/startUp.properties and specify values for the following options:

HttpPort=<port number>
HttpsPort=<port number>

Then restart the server, as described in Starting and stopping Striim.

3.10.3
Was this article helpful?
0 out of 0 found this helpful

Comments