The name of the challenge where Backups involve a large number of small Files, which stresses the Tsm Database due to the heavy updating and number of Database entries, and the Client's memory and processing power in performing an IncrementalBackup.
See DatabasePerformance for ways to mitigate the impact on the TsmDatabase and OptimizePerformance.
Other possible approaches:
To somewhat reduce Backup time, consider using -INCRBYDate backup, which eliminates getting a long list of files from the Server, massaging it in client memory, and then comparing as the FileSystem is traversed. (But see the INCRBYDate entry for side effects.)
Another Backup time reduction scheme: With some client FileSystems it may be known in what area updating occurs, as in the case of a company doing product testing which creates thousands of results files in subdirectories named by product and Date. Here you can tailor your backup to go directly at those Directories and skip the rest of the FileSystem, where you know that little or nothing has Changed.
- Journal-Based Backups may be a good alternative on Windows.
Consider 'dsmc BackupImage' (q.v.), to back up the physical image of a Volume (Raw LogicalVolume) rather than individually backing up the files within it.
- Some customers pre-combine many small files on the client system, as with the Unix 'tar' Command or personal computer File bundling packages, thus reducing the quantity to a single bundle File.
If regulations require you to keep files for a certain period, consider using BackupSets rather than doing Full backups.
Consider a "divide and conquer" approach, using parallel backup Processes to operate on separate areas of a FileSystem housing many small Files, to reduce the overall time to perform the backup. You may employ a 'dsmc i' for each major top-level directory, to back up into the same TsmServer FileSpace,
or use the VirtualMountPoint option to cause the FileSystem to be treated as multiple FileSpaces. Naturally, this can be effective only if your Disk and I/O Path can meet the demands.)
Q: would Using a DSMC process per unit of filesytem residing on a real separate physical volumes improve performance?
Your Retention policies need to be reasonable: don't arbitrarily retain a year's worth of versions, but rather keep as much as is really needed to recover files. Make sure you are running regular, unlimited expirations, else your TsmDatabase will balloon. The backup of small files is also problematic with TapeDrives with poor Start-stop characteristics (see Backhitch). The condition of the directory in which the small files exist can also slow things down: see "BackupPerformance". Consider turning on ClientTracing to identify the specific problem area.