Linux Shell Script to Archive Log Files by Flushing Out Content

by

You’re probably wondering how to archive log files, especially when your system is running all the time and constantly updating these logs.

Traditional Method: Usually, you’d move all the log files from their original location to an Archive directory and then clean up the original location. But there’s an issue—if your logs are continuously being updated by your application, moving them can cause errors in the application.

Solution: To avoid this issue, one way is to run the script during off-hours, taking 5-10 minutes of downtime. However, getting approval for downtime isn’t always easy, so here’s a better solution:

Trick: Instead of taking the system offline, you can copy all the files to another folder and then clear out the content of the log files in the original location. In this article you’ll learn Linux shell script to archive log files by flushing out content. 

If you prefer not to read all the details about this process, you can go straight to the script.

What Does the Script Do?

  1. COPY: It copies all your log files from the main LOG directory to the Archive directory.
  2. MAKE DIR: Before copying logs, it creates a new folder in the Archive directory and moves everything inside that folder.
  3. DELETE CONTENT: After copying, it clears the content of the log files in the original directory.
  4. CLEAN UP ARCHIVE: It removes log files in the Archive folder that are older than a set number of days (e.g., 7 days).

Directory Setup:

  • Your main LOG directory contains all the log files.
  • You should also have an Archive directory.
				cd /local/apps/WebServer/Web ls
Log Archive

Variables Used in the Script:

  • LEVROOT: The root directory. In this example, it’s "/local/apps/WebServer/Web".
  • LOG: The actual log directory, which is "${LEVROOT}/Log".
  • ARCHIVE: The archive directory, which is "${LEVROOT}/Archive".

You can organize your archive folder by creating a new folder (using the script) each day and then moving everything into the Archive subdirectory.

  • DATE: Holds today’s date.
  • TIME: Holds the current time.

Script Steps:

1. Define Variables:

				LEVROOT=/local/apps/WebServer/Web
DATE=$(date +%d%m%y)
TIME=$(date +%H%M)
ARCHIVE="$LEVROOT/Archive/$DATE-$TIME"
LOG="$LEVROOT/Log"

 

2. Delete Old Files: Find and delete files in the Archive directory that are older than 7 days (you can adjust the days as needed).

				find $LEVROOT/Archive/* -mtime +7 -type d -print -exec rm -R {} ;


# You can verify this by listing the files
find $LEVROOT/Archive/* -mtime +7

 

3. Create Archive Subdirectory: Create a subdirectory in the Archive directory to manage archived files.

				mkdir $ARCHIVE

 

4. Copy Log Files: Copy your log files into the Archive directory.

				find $LOG -type f -print -exec cp -p {} $ARCHIVE ;

 

5. Clear Log Files: Clear the content of log files in your main LOG directory.

				find $LEVROOT/*.* -type f -exec sh -c '> "{}"' ;

Linux/Unix Log Archive Script

Here is the complete files archiving script considering scenario discussed above. If you have any doubts in the script, then you may look at the scenario again or just leave your comment below. 
				#———————————————————————–
# Script for archiving logs by flushing out their content
# Script          : logs_Archive.sh
# 02-01-2025      : Linux/Unix Admin
#———————————————————————-
 

#!/bin/sh

# Variables Used:
LEVROOT=/local/apps/WebServer/Web
DATE=$(date +%d%m%y)
TIME=$(date +%H%M)
ARCHIVE="$LEVROOT/Archive/$DATE-$TIME"
LOG="$LEVROOT/Log"

# CLEAN up Archive directory

find $LEVROOT/Archive/* -mtime +7 -type d -print -exec rm -R {} ;

mkdir $ARCHIVE

# COPY files to archive directory

find $LOG/*.* -type f -print -exec cp -p {} $ARCHIVE ;

# Remove content from log files

find $LOG/*.* -type f -exec sh -c '> "{}"' ;

echo $ARCHIVE

echo $LOG

Summary

This script helps you archive log files without taking your system offline. It does this by copying the files to an Archive directory and then clearing the original files’ content. It also cleans up old archived files automatically. This approach prevents errors in applications that are actively writing to the log files.

ADDITIONAL RESOURCES: