Media Mover User's Guide

Media Mover provides accelerated file movement between Agents using standard templates.

Media Mover works on all Agents and provides support for growing source files.

Before you can transfer files using Media Mover, you must add and configure your jobs using one of four Media Mover job types. You must also configure any Agent groups for local storage.

Job Types

Media Mover offers four job types:

  • MediaDropBox
  • MediaAggregator
  • MediaDistributor
  • MediaReplicator

MediaDropBox

MediaDropBox transfers data between a single source location and one or more destinations. This job type polls the specified location at a user-determined interval and sends any changes detected to the specified destination(s).

MediaAggregator

MediaAggregator retrieves files from multiple Agents, transfers them to a single target Agent, and stores the files on its local disk or locally attached storage. The files transferred from the source Agents may be stored on a per Agent basis, or in a single flat folder on the target Agent. This is referred to as a pull file transfer template.

MediaDistributor

MediaDistributor allows you to schedule a push distribution in which multiple directories from a single source Agent are transferred to one or more target Agents.

MediaReplicator

MediaReplicator is a bulk file transfer option that allows you to schedule a distribution from a source Agent to one or more target Agents. As it is designed for dealing with large numbers of files, it does not have the file readiness checks that are present with MediaDropbox, MediaAggregator and MediaDistributor. This job type should be used when files are not growing in size in the source directory.

Configuring Media Mover Jobs

To add a Media Mover job:

  1. In your Manager, expand the Media Mover menu.
  2. Select MediaDropBox, MediaAggregator, MediaDistributor or MediaReplicator.
  3. Click Add.

To complete Basic configuration:

  1. Ensure that your selected job type appears in the Job Name field. If you choose to customize this name, it must begin with a letter and contain only underscores to separate words.
  2. In Drop Box/Source Specification, select the source Agent and its location.
  3. In Destination Specification, select the target Agent and its location.
  4. In Notification and Logging, specify the circumstances under which a notification e-mail or job failure report is sent, as well as their recipients.
  5. In Schedule, specify the frequency and timing of job runs.

To complete Advanced configuration:

Drop Box/Source Specification

  1. Enter file patterns (e.g. file extensions) to include or exclude. Separate multiple filters with commas. Include File Patterns includes all file names/types by default.

  2. Select whether to Exclude Subdirectories. Enter any specific subdirectories to exclude. Subdirectories that match those specified are excluded regardless of where they appear in the folder path. To exclude a folder at a specific level, use the @ symbol to anchor the folder path.

  3. For all job types except MediaReplicator, select the File Readiness Check Type:

    • None
    • Stream Growing Files: Transfers in-progress files before completion. Files are monitored constantly and when any data is discovered, it is transferred. This option is particularly useful when supporting the delivery of large files.
    • Consecutive Seek: Recommended. Reads the last few bytes of the file. After a specified number of delay, reads the same bytes again. If the content matches, the file is sent.
    • MD5 Hash: Uses an MD5 Hash to compute a hash on files. After a specified delay, the hashes are recomputed. Only files with identical hashes are transferred. Job initialization may take a long time for large files. This option can mitigate performance issues encountered when using MD5 Hash as a file readiness check type.
    • Date/Time and Size: Uses a difference in date/time or file size to compare readiness. Files that have identical date/time and size are transferred.

    Note: If the local or network file system supports file locking, the readiness check is not required.

    If File Readiness Check Type is set to Consecutive Seek, MD5 Hash, or Date/Time and Size, set the File Readiness Check Delay. This is the pause, in seconds, between two phases of file readiness check. The default delay is 10 seconds. When set to Stream Growing Files, this is the time that must pass without file changes for the file to be declared complete.

    If File Readiness Check Delay is set to Stream Growing Files, specify the following:

    • Header Check Delay: The time in seconds that the transfer waits after the end of the file is detected before transferring the file based on the size specified in Header Rewrite Size. The default is 10 seconds.
    • Header Rewrite Size: The number of bytes to transfer in the header file. The default is 1 MB.
  4. Specify any Source Deletions After Successful Transfer and enter any Source Directories to Exclude from Deletion Scan. These options allows the removal of specific files or directories after a successful transfer. Empty directories will only be deleted if they have not been modified within the previous five minutes.

  5. Specify whether to Skip Source File Not Found On Send and/or Skip Files With 0 Byte Size.

  6. Select any options by which to filter Source File Data. This option allows you to filter files by date or time, based on when the file was last modified, or accessed, or when the file was created.

Destination Specification

Note: Destination Specification options are not available for MediaReplicator.

  1. Select Directory Mapping Options to be added to the target directory path during the transfer. If neither item is checked, all files will be placed in a single parent folder.
  2. Choose whether to Flatten Directory Structure. Flattening your directory structure will write all transferred files to the destination directory without using the folder structure from the origin.
  3. If you choose Job Date/Time, select the Maximum Job Date-Time Directories to maintain on the target Agent. The default is 10. When the value is exceeded, the oldest job date/time directory is deleted.
  4. Specify whether to Remove Expired Files/Directories on Target and set the Expiration Period, which defaults to 30 days.

Transport Options

  1. Select the number of Simultaneous Transfer Streams. The default is 10.

  2. Specify to Use UDP Control Channel instead of TCP. This option should be used for firewall traversal when the TCP port cannot be opened.
    Note: When Use UDP Control Channel is set to Yes, file transfers automatically use UDP WAN acceleration, which may override the Use WAN Accelerator setting.

  3. Select whether to Use WAN Accelerator. The default is Yes. This setting uses all available bandwidth up to the user-specified bandwidth maximum. Use this option on high-latency, high-bandwidth networks where throughput is a top priority. If the throughput is less than 57200 Bps, the transfer will switch to TCP.

  4. Set the WAN Accelerator Aggressiveness, which indicates the job's sensitivity to other network traffic. The higher the level of aggressiveness, the less the job accommodates other network traffic.

  5. Enter any of the following bandwith settings:

    • Bandwidth Ceiling: The maximum throttle rate for the transfer. Typically, this is the maximum speed of the network. Enter 0 to enable auto-detection for the best bandwidth rate.
    • Bandwidth Floor: The minimum rate at which data should be sent. Enter 0 to enable auto-detection for the best bandwidth rate. If WAN Accelerator Aggressiveness is set to Medium or Low, the rate specified here will be used.
    • Bandwidth Throttle by Time of Day: The bandwidth used according to the time of day and/or day of the week. Click + to add additional time parameters, or click x to remove them.

    Note: A bandwidth throttle specifies the rate at which the source will make the data available to the network, regardless of how fast the machine is sending. Bandwidth Throttle is the rate at which the source reads the data from disk and puts it in the send queue. A throttle is split amongst the concurrent target Agents. If, for example, your throttle is 50Mb/sec and you specify two target Agents, the bandwidth be divided equally at 25Mb/sec per target.

    Note: Once a job has started, all bandwidth throttles are applied at the times based on the Daylight Savings Time (DST) in effect when the job started. If DST changes while the job is running, bandwidth time of day changes may be off by an hour.

Advanced Options

  1. Set the Encryption Level. The default is No Encryption – unsigned:

    • Encryption On: Uses the following maximum or high encryption level - Cipher=AES256-SHA, Keysize=256/256.
    • No encryption – signed: Transfers unencrypted data, but includes the SSL protocol's message digest calculation and signing to ensure data stream integrity.
    • No encryption – unsigned: After the initial SSL authentication of the endpoints, this setting allows the transfer of data to proceed with no encryption, no message digest computation, and no signing of the message digests. This mode of operation is only for raw performance, since it makes no guarantee of the integrity of the data stream other than the default one has with a normal network channel. The underlying TCP protocol can guarantee the integrity of messages across each single network hop, but has no facilities for detecting a man-in-the-middle attack.

    Note: Mutual authentication is always used, regardless of the encryption level specified. Encryption is done in-stream and not on disk prior to sending.

  2. Select the Delivery Mode for transferring files:

    • Fast: Does not create temporary work files. This mode is optimized for speed.
    • Normal: Uses temporary work files during transfer. Temporary files allow an interrupted transfer to resume from where the interruption occurred. The creation of temporary files adds time to the transfer, but provides resiliency.
    • Log File Name: Transfers log file names with files.
    • Certify File Delivery: Creates an Agent certified delivery log of files transferred.
  3. Choose whether to Compress Files. The default value is No. Choose this option if you are running over a low speed WAN link.

  4. Specify whether to Follow Symbolic Links. If a symbolic link references a directory and this option is enabled, the directory is transferred. If disabled, only the symbolic link itself is transferred.

  5. If you are running over a low speed WAN link, enable Incremental Transfer to transfer only the changed bytes of a file, rather than the entire file.

  6. In MediaReplicator only, enable Synchronize Target with Source to delete files from the target folder if they are not found in the source folder.
    Note: Use caution when enabling the Synchronize Target with Source option. Any content deleted on the target Agent is not recoverable via Signiant.

  7. Select whether to Verify for sufficient disk space before Job transfers start and/or Verify if target directory exists. These verifications ensure that the transfer is not aborted due to insufficient disk space or a nonexistent directory.

Notification And Logging

  1. Select the Number of consecutive failures before notification is sent. The default is 1.
  2. Select the Log Detail Level. Debug provides the greatest level of detail, Error provides the least detail. A lower level of log detail will prevent performance delays.

SNMP

  1. Enter SNMP Trap Receivers. These are hostnames and/or IP addresses of systems that are listening for SNMP traps.
  2. Enter the SNMP Trap Community String, i.e. the password to be sent in the trap.
  3. Select SNMP Trap Types. These are the job completion statuses on which SNMP traps are sent.

Schedule

Select additional time parameters, choose whether a transfer is interrupted upon failure, and set the job's queuing priority.