Environment Optimization

Manager+Agents 14.0 introduces new features aimed at improving overall performance of your Manager including:

• New REST interface architecture
• New scheduler architecture
• Settings to improve resource controlled job performance
• Database optimizations

These improvements, initially included as opt-in features for Manager+Agents 13.5, address issues related to performance based on job performance and overall processing load when using the Manager.

New REST Interface Architecture

By default, using the REST API relies on your custom integration to manage loads on the REST interface. Prior to 13.5, jobs would occasionally fail due to lack of available resources in the REST interface.

With the new interface, the REST API queue uses a Token Bucket algorithm to balance requests against a set number of resource tokens.

You can now execute over 100 concurrent REST requests with minimal impact to overall performance.

New Scheduler Architecture

Job Evaluator improvements have been made to allow for better performance for large databases with a high amount of job runs.

This change dramatically improves responsiveness to tasks with a high level of concurrency and increases performance from within the Manager application, especially with regard to displaying large amounts of data at a time.

Manual Optimization

While Manager+Agents 14.0 includes a number of features that automatically increase performance, you can further optimize your Manager by setting parameters that improve the management of large numbers of jobs.

High Concurrency

Manager installations include command line tools that allow you to send override commands to an Agent's configuration using the dds_admin application.

Increasing Resource Control Timeout

When submitting a large number of jobs concurrently, or with a very small interval between them, you can prevent Agent based resource controls causing a timeout error by increasing the timeout value for the resource control service.

Configure Agent timeout values by setting the configuration via the command line:

> dds_admin -one_shot "set RESBRKR_RESPONSE_TIMEOUT 60" customer*Admin agent.example.com

Note: You can also run set RESBRKR_RESPONSE_TIMEOUT 60 from the command line on the Agent.

Increasing Port Range

If you expect to run more than 100 jobs at a time, you must to increase the inbound and outbound port ranges from 100 to 300 to allow your Agents to handle higher capacity.

Configure the UDP port range by setting the configuration via the command line:

1. Log into the Manager via command line.
2. Run dds_admin.
3. At the dds_admin prompt, set the UDP destination port range:

>set UDP_DEST SIZE=300

4. Set the UDP origin port range:

>set UDP_ORIG SIZE=300

5. Quit dds_admin by entering the q command.

Note: You may also need to work with your firewall administrator to allow access through additional ports as needed (e.g. from 49221-49321 to 49221-49531).

Automatic Job Deletion

To preserve your database's performance, you can choose to automatically delete jobs once they have run by adding parameters to your signiant.ini file. You can choose to remove jobs using soft or hard deletion.

A soft deletion removes the job from the Manager, but allows you to query the job history directly from the database before it is removed by a maintenance job.

A hard deletion immediately removes the job record from the database, including its history and record of files included in the job.

To remove a job from the database after it is successfully completed, set SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS in your signiant.ini file to your desired level:

• SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS=off - Do not automatically delete jobs.
• SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS=soft - Automatically delete jobs from the Manager after a successful completion, and queue them for removal from the database during the next maintenance job.
• SCHDSVR_JOBOPT_AUTODELETE_IF_SUCCESS=hard - Immediately remove jobs from the database after a successful completion.

To remove a job from the database regardless of its completion status, set SCHDSVR_JOBOPT_AUTODELETE in your signiant.ini to your desired level:

• SCHDSVR_JOBOPT_AUTODELETE=off - Do not automatically delete jobs.
• SCHDSVR_JOBOPT_AUTODELETE=soft - Automatically delete jobs from the Manager after a job run, and queue them for removal from the database during the next maintenance job.
• SCHDSVR_JOBOPT_AUTODELETE=hard - Immediately remove jobs from the database after a job run.

Increasing Agent System Processes

Linux operating systems allow you to manually override the number of processes available to the kernel, allowing improved Agent performance when running a job.

Note: Increasing the number of available processes beyond the system's capacity can cause your system to stop responding.

To see the number of processes available to the kernel, use the sysctl command:

$ sysctl kernel.pid_max

To set the number of processes set the kernel.pid_max:

$ sysctl kernel.pid_max = <number of processes>