Environment Optimization

Manager+Agents 13.5 introduces new opt-in features aimed at improving overall performance of your Manager including:

  • New REST interface architecture
  • New scheduler architecture
  • Settings to improve resource controlled job performance
  • Database optimizations

These improvements address issues related to performance based on job performance and overall processing load when using the Manager. These changes strongly improve Manager performance, but must be manually enabled via your configuration files.

Enabling the new REST interface architecture

By default, using the REST API relies on your custom integration to manage loads on the REST interface. Prior to 13.5, jobs would occasionally fail due to lack of available resources in the REST interface.

With the new interface, the REST API queue uses a Token Bucket algorithm to balance requests against a set number of resource tokens.

You can now execute over 100 concurrent REST requests with minimal impact to overall performance by enabling the new feature.

Manager 13.5 Job Execution Performance

To enable the new REST interface:

  1. Create a backup of your signiant.ini file.

  2. Open signiant.ini and add the following lines to the file:

    # Token Bucket Architecture Opt-in
    BUCKET_CAPACITY=500
  3. Save and close your signiant.ini file.

  4. Restart Manager's JBOSS service using a service manager.

Enabling the new Scheduler architecture

Job Evaluator improvements have been made to allow for better performance for large databases with a high amount of job runs.

This change dramatically improves responsiveness to tasks with a high level of concurrency and increases performance from within the Manager application, especially with regard to displaying large amounts of data at a time.

Manager 13.5 Job View Load Time

To enable the new scheduler architecture:

  1. Create a backup of your signiant.ini file.

  2. Open signiant.ini and add the following lines to the file:

    # Java Evaluator Opt-in
    SCHDSVR_EVALUATE_JOBS=no
  3. Save and close your signiant.ini file.

  4. Restart Manager's JBOSS service using a service manager.

Optimizing Interface Performance for Resource Controlled Jobs

As part of its running environment, Signiant Manager uses a run control file to set how the background Java Virtual Machine (JVM) runs according to your /usr/signiant/dds/init/.siginitrc settings file.

If you are running a high number of jobs and encounter an OutOfMemory error, you can increase the memory allocation pools for your JVM and rule server JVM.

To increase your JVM memory allocation:

  • Set JAVA_JBOSS_SERVER_OPTS to -Xms1024m -Xmx4096m
  • Set JAVA_DB_SERVER_OPTS to -Xmx2048m

Note: Do not modify any other settings in the .siginitrc file unless otherwise instructed.

High Concurrency

If you expect to run more than 100 jobs at a time, you must to increase the inbound and outbound port ranges from 100 to 300 to allow your Agents to handle higher capacity.

This change is made in your DDS configuration file:

  • Linux: dds.conf
  • Windows: dds.cfg

The configuration file sets the default Origin UDP port range for inbound traffic and Destination UDP port range to 100 in the configuration file:

Origin UDP port range is 100
Destination UDP port range is 100

Replace the values with 300 and add a comment to each line to easily find your customization if needed:

Origin UDP port range is 300  #sets the limit for inbound traffic
Destination UDP port range is 300 #sets the limit for outbound traffic

Note: You may also need to work with your firewall administrator to allow access through additional ports as needed (e.g. from 49221-49321 to 49221-49531).

Database Optimization

Signiant Manager uses a PostgreSQL server as part of its operational environment, and is configured for maximum compatibility for system requirements. If your system surpasses the minimum requirements, you can optimize your database for higher capacity.

If your workflow does not require data retention, you can choose to automatically delete jobs from your database to increase overall performance.

Increasing Database Operability

Increasing your system's overall capacity to connect and interact with the database can be useful when working with a large number of jobs under resource control.

To increase the number of concurrent jobs you can run at the same time, you must increase your database's maximum active connection count by adding the following line to your signiant.ini:

DBMAXACTIVECONN=150

Additional database settings can be modified using the /usr/signiant/dds/db/pgsql/conf/postgresql.conf file.

The Connections and Authentication section of the configuration file contains the max_connections setting.

  • For optimal performance, increase your max_connections to 400.

The Lock Management section contains settings to manage a database's ability to preserve database values while in operation.

  • For increased performance, set max_locks_per_transaction to 2000.

The Resource Usage section controls how your database uses system memory to use when performing database operations.

  • Depending on your system's available resources, you can increase the shared_buffers to 1024MB.

Note: Large-scale organizations with sufficient hardware capacity can increase the max_locks_per_transaction setting up to 40960, and shared_buffers up to 8026MB.

Automatic Job Deletion

To preserve your database's performance, you can choose to automatically delete jobs once they have run by adding parameters to your signiant.ini file. You can choose to remove jobs using soft or hard deletion.

  • A soft deletion removes the job from the Manager, but allows you to query the job history directly from the database before it is removed by a maintenance job.

  • A the hard setting immediately removes the job record from the database, including its history and record of files included in the job.

To remove a job from the database after it is successfully completed, set SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS in your signiant.ini file to your desired level:

  • SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS=off - Do not automatically delete jobs.
  • SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS=soft - Automatically delete jobs from the Manager, and queue them for removal from the database during the next maintenance job.
  • SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS=hard - Immediately remove jobs from the database.

Post-Upgrade Optimization

During a version upgrade, some field values are preserved, while others are overwritten. Settings that are overwritten must be re-applied to maintain optimization.

All Signiant.ini settings are preserved on upgrade.

All Postgresql.conf values are overwritten, with the exception of those in three fields:

  • listen_addresses
  • log_directory
  • maxlocksper_transaction

All other customization must be re-applied after upgrade.