Manager+Agents 13.5 introduces new opt-in features aimed at improving overall performance of your Manager including:
These improvements address issues related to performance based on job performance and overall processing load when using the Manager. These changes strongly improve Manager performance, but must be manually enabled via your configuration files.
By default, using the REST API relies on your custom integration to manage loads on the REST interface. Prior to 13.5, jobs would occasionally fail due to lack of available resources in the REST interface.
With the new interface, the REST API queue uses a Token Bucket algorithm to balance requests against a set number of resource tokens.
You can now execute over 100 concurrent REST requests with minimal impact to overall performance by enabling the new feature.
To enable the new REST interface:
Open signiant.ini and add the following lines to the file:
# Token Bucket Architecture Opt-in BUCKET_CAPACITY=500
Job Evaluator improvements have been made to allow for better performance for large databases with a high amount of job runs.
This change dramatically improves responsiveness to tasks with a high level of concurrency and increases performance from within the Manager application, especially with regard to displaying large amounts of data at a time.
To enable the new scheduler architecture:
Open signiant.ini and add the following lines to the file:
# Java Evaluator Opt-in SCHDSVR_EVALUATE_JOBS=no
As part of its running environment, Signiant Manager uses a run control file to set how the background Java Virtual Machine (JVM) runs according to your
/usr/signiant/dds/init/.siginitrc settings file.
If you are running a high number of jobs and encounter an
OutOfMemory error, you can increase the memory allocation pools for your JVM and rule server JVM.
To increase your JVM memory allocation:
Note: Do not modify any other settings in the .siginitrc file unless otherwise instructed.
Manager installations include command line tools that allow you to send override commands to an Agent's configuration using the
When submitting a large number of jobs concurrently, or with a very small interval between them, you can prevent Agent based resource controls causing a timeout error by increasing the timeout value for the resource control service.
Configure Agent timeout values by setting the configuration via the command line:
> dds_admin -one_shot "set RESBRKR_RESPONSE_TIMEOUT 60" customer*Admin agent.example.com
Note: You can also run
set RESBRKR_RESPONSE_TIMEOUT 60 from the command line on the Agent.
If you expect to run more than 100 jobs at a time, you must to increase the inbound and outbound port ranges from 100 to 300 to allow your Agents to handle higher capacity.
Configure the UDP port range by setting the configuration via the command line:
dds_admin prompt, set the UDP destination port range:
>set UDP_DEST SIZE=300
Set the UDP origin port range:
>set UDP_ORIG SIZE=300
dds_adminby entering the
Note: You may also need to work with your firewall administrator to allow access through additional ports as needed (e.g. from 49221-49321 to 49221-49531).
Signiant Manager uses a PostgreSQL server as part of its operational environment, and is configured for maximum compatibility for system requirements. If your system surpasses the minimum requirements, you can optimize your database for higher capacity.
If your workflow does not require data retention, you can choose to automatically delete jobs from your database to increase overall performance.
Increasing your system's overall capacity to connect and interact with the database can be useful when working with a large number of jobs under resource control.
To increase the number of concurrent jobs you can run at the same time, you must increase your database's maximum active connection count by adding the following line to your signiant.ini:
Additional database settings can be modified using the
The Connections and Authentication section of the configuration file contains the
The Lock Management section contains settings to manage a database's ability to preserve database values while in operation.
The Resource Usage section controls how your database uses system memory to use when performing database operations.
Note: Large-scale organizations with sufficient hardware capacity can increase the
max_locks_per_transaction setting up to 40960, and
shared_buffers up to 8026MB.
To preserve your database's performance, you can choose to automatically delete jobs once they have run by adding parameters to your signiant.ini file. You can choose to remove jobs using soft or hard deletion.
A soft deletion removes the job from the Manager, but allows you to query the job history directly from the database before it is removed by a maintenance job.
A hard deletion immediately removes the job record from the database, including its history and record of files included in the job.
To remove a job from the database after it is successfully completed, set
SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS in your signiant.ini file to your desired level:
SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS=off- Do not automatically delete jobs.
SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS=soft- Automatically delete jobs from the Manager after a successful completion, and queue them for removal from the database during the next maintenance job.
SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS=hard- Immediately remove jobs from the database after a successful completion.
To remove a job from the database regardless of its completion status, set
SCHDSVR_JOBOPT_AUTODELETE in your signiant.ini to your desired level:
SCHDSVR_JOBOPT_AUTODELETE=off- Do not automatically delete jobs.
SCHDSVR_JOBOPT_AUTODELETE=soft- Automatically delete jobs from the Manager after a job run, and queue them for removal from the database during the next maintenance job.
SCHDSVR_JOBOPT_AUTODELETE=hard- Immediately remove jobs from the database after a job run.
During a version upgrade, some field values are preserved, while others are overwritten. Settings that are overwritten must be re-applied to maintain optimization.
All Signiant.ini settings are preserved on upgrade.
All Postgresql.conf values are overwritten, with the exception of those in three fields:
All other customization must be re-applied after upgrade.