Linux Shell Script to Archive Log Files

by

Managing log files is a crucial task for Linux and Unix administrators. One of the most effective methods for handling these files involves setting up a script that automates the process of archiving logs.

While there are numerous ways to achieve this, finding a solution that is both robust and adaptable to various scenarios can be challenging.

However, this article will present a highly efficient and versatile method that simplifies the management of log files.

If you prefer to skip the detailed explanation of the archiving process, you can jump directly to the log archive script section.

For those who want to understand the process more deeply, let’s walk through a common scenario where this script is particularly useful.

Scenario Overview

Imagine you have a directory where all your log files are stored, and you want to move these files to an archive directory. Here’s how you can automate this process:

  1. Move Logs to Archive Directory: Before moving your log files, create a new subdirectory under the archive directory for the current date and time. This organization helps in managing archived logs efficiently.

  2. Remove Old Logs: Delete log files that are older than a specified number of days (e.g., 7 days). This ensures that your log directories do not become cluttered with outdated files.

By automating this process, you can effectively manage your log files without the need for manual intervention on a weekly or monthly basis.

Setting Up Your Environment

Consider the following directory structure for your logs:

  • LOG Directory: This is where all your current log files are stored.
  • Archive Directory: This is where your archived logs will be moved.

    Here is an example of how your directory might be set up:

				cd /local/apps/BatchServer/MyDIStudioJob1 ls

Log      Archive

LOG=/local/apps/BatchServer/MyDIStudioJob1/Log

Archive=/local/apps/BatchServer/MyDIStudioJob1/Archive

You should see two directories: Log and Archive.

Define the following variables in your script:

  • LEVROOT: The root directory of your logs and archives. In this example, it is /local/apps/BatchServer/MyDIStudioJob1.
  • LOG: The directory where your current logs are stored. For this example, it is $LEVROOT/Log.
  • ARCHIVE: The directory where your logs will be archived. For this example, it is $LEVROOT/Archive.

Script Variables

Here are the variables used in the script:

  • LEVROOT: The root directory, e.g., /local/apps/BatchServer/MyDIStudioJob1.
  • DATE: A variable to hold the current date in the format ddmmyy.
  • TIME: A variable to hold the current time in the format HHMM.
  • ARCHIVE: A variable to define the path for the new archive subdirectory, e.g., $LEVROOT/Archive/$DATE-$TIME.
  • LOG: The path to the log directory, e.g., $LEVROOT/Log.

Step-by-Step Script

1. Define Variables

				LEVROOT=/local/apps/BatchServer/MyDIStudioJob1
DATE=$(date +%d%m%y)
TIME=$(date +%H%M)
ARCHIVE="$LEVROOT/Archive/$DATE-$TIME"
LOG="$LEVROOT/Log"

 

2. Search and Delete Old Files

Use the find command to search for files in the archive directory that are older than 7 days and remove them. Adjust the number of days as needed.

				find $LEVROOT/Archive/* -mtime +7 -type d -print -exec rm -R {} \;

To verify which files will be deleted, you can list them first:

				find $LEVROOT/Archive/* -mtime +7

 

3. Create a Subdirectory

Create a new subdirectory within the archive directory to store the logs for the current date and time.

				mkdir $ARCHIVE

 

4. Move Log Files

Move the log files from the LOG directory to the newly created ARCHIVE subdirectory.

				find $LOG -type f -print -exec mv {} $ARCHIVE \;

By following these steps, you can effectively manage your log files with minimal manual effort. This script ensures that old logs are cleaned up, and new logs are archived in an organized manner.

Linux Log Archiving Script

This script handles the task of archiving log files from a specified directory. It creates a new subdirectory in the archive folder for each execution based on the current date and time. It also removes old archive directories that are beyond a specified age to prevent clutter. 

				#!/bin/sh

#—————————————————————–
# Script for archiving logs
# Script          : logs_Archive.sh
# Created on      : 02-01-2025
# Author           : Linux/Unix Admin
#—————————————————————–

# Variables Used:
LEVROOT="/local/apps/BatchServer/MyDIStudioJob1"     # Root directory
DATE=$(date +%d%m%y)                               # Current date in ddmmyy format
TIME=$(date +%H%M)                                 # Current time in HHMM format
ARCHIVE="$LEVROOT/Archive/$DATE-$TIME"             # Archive subdirectory path
LOG="$LEVROOT/Log"                                # Log directory path

# Clean up old archives older than 7 days
find "$LEVROOT/Archive" -mindepth 1 -maxdepth 1 -mtime +7 -type d -print -exec rm -R {} \;

# Create a new subdirectory for today's logs
mkdir -p "$ARCHIVE"

# Move all log files from LOG directory to the new ARCHIVE subdirectory
find "$LOG" -type f -print -exec mv {} "$ARCHIVE" \;

# Output the paths for verification
echo "Logs have been archived to: $ARCHIVE"
echo "Log directory: $LOG"

Script Breakdown

  • LEVROOT: This variable sets the root directory for both the log files and the archive. In your script, it is defined as /local/apps/BatchServer/MyDIStudioJob1.

  • DATE: This variable captures the current date formatted as ddmmyy (e.g., 150824 for August 15, 2024). This helps in organizing archived logs by date.

  • TIME: This variable captures the current time formatted as HHMM (e.g., 1234 for 12:34 PM). This ensures that each archive subdirectory has a unique timestamp.

  • ARCHIVE: This variable defines the path to the new subdirectory where the logs will be archived. It combines LEVROOT, DATE, and TIME to create a path like /local/apps/BatchServer/MyDIStudioJob1/Archive/150824-1234.

  • LOG: This variable specifies the path to the directory containing the log files that need to be archived. It is set to the root directory path (/local/apps/BatchServer/MyDIStudioJob1/).

  • find $LEVROOT/Archive/* -mtime +7 -type d -print -exec rm -R {} \;: This command finds and deletes directories within the Archive directory that are older than 7 days. The -mtime +7 flag specifies files modified more than 7 days ago, and -type d ensures only directories are targeted. The -exec rm -R {} \; part removes these directories.

  • mkdir $ARCHIVE: This command creates a new subdirectory for today’s archived logs. The -p flag is omitted here but can be used to avoid errors if the directory already exists.

  • find $LOG -type f -print -exec mv {} $ARCHIVE \;: This command finds all files in the LOG directory and moves them to the newly created ARCHIVE subdirectory. The -type f flag ensures that only files are moved, not directories.

  • echo $ARCHIVE and echo $LOG: These commands print the paths of the ARCHIVE and LOG directories, providing a simple verification that the script has run and showing the directories involved.

This breakdown helps in understanding each part of the script and its purpose in the log archiving process. That’s it. This is all about Linux shell script to archive log files. 

ADDITIONAL RESOURCES: