Linux/Unix Parameterized Shell Script to Archive logs

by

You might already be familiar with the traditional, but more effective, log archiving script. If not, you may want to check out that script first.

If you have several different jobs running, each creating large files in its own directory, you can modify the script slightly so it can be used for all these jobs instead of creating multiple scripts.

Parameterized UNIX/Linux Shell Script

A simple way to reuse your script is by passing parameters from outside the script. You provide the parameter(s) when running the script.

For example, let’s say you have multiple directories under /local/apps/BatchServer/. You can make the script generic by passing the “directory name” as a parameter. Suppose you have subdirectories like "MyDIStudioJob_1", "MyDIStudioJob_2", "MyDIStudioJob_3", etc. You can pass these directory names from outside, and they will be referred to as $1 in the script.

Now, your "LEVROOT" will become:

LEVROOT=/local/apps/BatchServer/$1

PS- Make sure you have Log and Archive folders created under each sub-directory to demonstrate this sample example.

LOG ARCHIVING SCRIPT

				#——————————————————
# Script for archiving logs
# Script         : logs_Archive.sh
# Parameter   : $1: Sub-directory Name
# 02-01-2019 : Linux/Unix Admin
#——————————————————

#!/bin/sh

# Variables Used:
LEVROOT=/local/apps/Batchserver/$1    # Base directory for the logs, based on the sub-directory passed as a parameter
DATE=$(date +%d%m%y)                  # Current date in the format DDMMYY
TIME=$(date +%H%M)                    # Current time in the format HHMM
ARCHIVE="$LEVROOT/Archive/$DATE-$TIME" # Path to the archive directory with date and time stamp
LOG="$LEVROOT/"                       # Path to the log directory

# Clean up old archives
find $LEVROOT/Archive/* -mtime +7 -type d -print -exec rm -R {} ; # Find and remove archive directories older than 7 days

# Create a new archive directory
mkdir $ARCHIVE                       # Create a new directory for the current archive

# Move log files to the archive directory
find $LOG -type f -print -exec mv {} $ARCHIVE ; # Find all files in the log directory and move them to the archive

# Output the paths for verification
echo $ARCHIVE                         # Print the archive directory path
echo $LOG                             # Print the log directory path
  1. LEVROOT:
    • This variable sets the root directory for the logs, which is determined by the sub-directory name passed as an argument ($1) when running the script.
  2. DATE and TIME:
    • These variables capture the current date and time, respectively, which are used to create unique archive directory names. This helps in organizing the archived logs by when they were archived.
  3. ARCHIVE:
    • This variable defines the full path for the archive directory, including the date and time stamp, ensuring that each archive session is stored in a separate directory.
  4. LOG:
    • This is the path to the log directory where the current log files are located.
  5. Cleaning Up Old Archives:
    • The find command is used to identify and delete any directories in the Archive folder that are older than 7 days. This helps in managing disk space by removing outdated logs.
  6. Creating a New Archive Directory:
    • The mkdir command creates a new directory under the Archive folder using the current date and time, providing a unique location for each archiving session.
  7. Moving Log Files:
    • The script moves all files from the log directory to the newly created archive directory using the find and mv commands. This effectively archives the logs while clearing the log directory for new entries.
  8. Output Verification:
    • Finally, the script echoes the paths of the archive and log directories to the console, allowing the user to verify where the logs have been archived and which directory was cleared.

This script efficiently handles log archiving by organizing files based on the date and time of archiving, while also cleaning up old archives to conserve disk space.

EXECUTING THE SCRIPT

If you want to run this script for job number 1, which has logs generated under the subdirectory "MyDIStudioJob_1", you can do so by running:

				./logs_Archive.sh MyDIStudioJob_1  # run script with parameter

 

Or, if you want to run it for job number 3:

				./logs_Archive.sh MyDIStudioJob_3  # run script with parameter

Important Note: This script moves log files into the Archive and cleans up the original directory. However, if you have a job or application running 24/7, moving your log files might not be an option. For handling such scenarios, you may want to explore other solutions, as discussed in this blog post.

ADDITIONAL RESOURCES: