A few quick thumb rules when writing shell scripts
Shell Script Best Practices
BASH Frequently Asked Questions
https://mywiki.wooledge.org/BashFAQ
Bash Pitfalls
https://mywiki.wooledge.org/BashPitfalls
Safe ways to do things in bash
https://github.com/anordal/shellharden/blob/master/how_to_do_things_safely_in_bash.md
Better Bash Scripting in 15 Minutes
https://robertmuth.blogspot.com/2012/08/better-bash-scripting-in-15-minutes.html
Writing Robust Bash Shell Scripts
https://www.davidpashley.com/articles/writing-robust-shell-scripts/
Shell Field Guide
https://raimonster.com/scripting-field-guide/
There’s also a BASH3 Boilerplate
https://bash3boilerplate.sh/
Run cat -v file.sh
There is most likely a carriage return or no-break space in the file. cat -v will show them as ^M and M-BM- or M- respectively. It will similarly show any other strange characters in the file.
Remove the Windows line breaks with
tr -d '\r' < file.sh > fixedfile.sh
Example of error message when the script seems ok
/debugshell/Untitled-1.sh: line 10: syntax error near unexpected token 'done'
/debugshell/Untitled-1.sh: line 10: 'done'
OR
Use the ShellCheck extension in VSCode.
This script will be useful to analyze the disk usage and if the reported disk space is more than 90 % an email will be sent to the administrator.
#!/bin/sh
# set -x
# Shell script to monitor or watch the disk space
# It will send an email to $ADMIN, if the (free available) percentage of space is >= 90%.
# ------------------------------------------------------------------------------------------
# Set admin email so that you can get email.
ADMIN="root"
# set alert level 90% is default
ALERT=90
# Exclude list of unwanted monitoring, if several partions then use "|" to separate the partitions.
# An example: EXCLUDE_LIST="/dev/hdd1|/dev/hdc5"
EXCLUDE_LIST="/auto/ripper"
#
#:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
#
main_prog() {
while read -r output;
do
#echo $output
usep=$(echo "$output" | awk '{ print $1}' | cut -d'%' -f1)
partition=$(echo "$output" | awk '{print $2}')
if [ "$usep" -ge $ALERT ] ; then
echo "Running out of space \"$partition ($usep%)\" on server $(hostname), $(date)" | \
mail -s "Alert: Almost out of disk space $usep%" $ADMIN
fi
done
}
if [ "$EXCLUDE_LIST" != "" ] ; then
df -H | grep -vE "^Filesystem|tmpfs|cdrom|${EXCLUDE_LIST}" | awk '{print $5 " " $6}' | main_prog
else
df -H | grep -vE "^Filesystem|tmpfs|cdrom" | awk '{print $5 " " $6}' | main_prog
fi
This script will do the incremental backup into an external mounted hard-drive. It is to take a backup of the /home directory. However, it can be modified to suit the requirements.
#!/bin/bash
# ----------------------------------------------------------------------
# mikes handy rotating-filesystem-snapshot utility
# ----------------------------------------------------------------------
# this needs to be a lot more general, but the basic idea is it makes
# rotating backup-snapshots of /home whenever called
# ----------------------------------------------------------------------
unset PATH # suggestion from H. Milz: avoid accidental use of $PATH
# ------------- system commands used by this script --------------------
ID=/usr/bin/id;
ECHO=/bin/echo;
MOUNT=/bin/mount;
RM=/bin/rm;
MV=/bin/mv;
CP=/bin/cp;
TOUCH=/bin/touch;
RSYNC=/usr/bin/rsync;
# ------------- file locations -----------------------------------------
MOUNT_DEVICE=/dev/hdb1;
SNAPSHOT_RW=/root/snapshot;
EXCLUDES=/usr/local/etc/backup_exclude;
# ------------- the script itself --------------------------------------
# make sure we're running as root
if (( $($ID -u) != 0 )); then { $ECHO "Sorry, must be root. Exiting..."; exit; } fi
# attempt to remount the RW mount point as RW; else abort
$MOUNT -o remount,rw $MOUNT_DEVICE $SNAPSHOT_RW ;
if (( $? )); then
{
$ECHO "snapshot: could not remount $SNAPSHOT_RW readwrite";
exit;
}
fi
# rotating snapshots of /home (fixme: this should be more general)
# step 1: delete the oldest snapshot, if it exists:
if [ -d $SNAPSHOT_RW/home/hourly.3 ] ; then $RM -rf $SNAPSHOT_RW/home/hourly.3 ; fi;
# step 2: shift the middle snapshots(s) back by one, if they exist
if [ -d $SNAPSHOT_RW/home/hourly.2 ] ; then $MV $SNAPSHOT_RW/home/hourly.2 $SNAPSHOT_RW/home/hourly.3 ; fi;
if [ -d $SNAPSHOT_RW/home/hourly.1 ] ; then $MV $SNAPSHOT_RW/home/hourly.1 $SNAPSHOT_RW/home/hourly.2 ; fi;
# step 3: make a hard-link-only (except for dirs) copy of the latest snapshot,
# if that exists
if [ -d $SNAPSHOT_RW/home/hourly.0 ] ; then $CP -al $SNAPSHOT_RW/home/hourly.0 $SNAPSHOT_RW/home/hourly.1 ; fi;
# step 4: rsync from the system into the latest snapshot (notice that
# rsync behaves like cp --remove-destination by default, so the destination
# is unlinked first. If it were not so, this would copy over the other
# snapshot(s) too!
$RSYNC -va --delete --delete-excluded --exclude-from="$EXCLUDES" /home/ $SNAPSHOT_RW/home/hourly.0 ;
# step 5: update the mtime of hourly.0 to reflect the snapshot time
$TOUCH $SNAPSHOT_RW/home/hourly.0 ;
# and thats it for home.
# now remount the RW snapshot mountpoint as readonly
$MOUNT -o remount,ro $MOUNT_DEVICE $SNAPSHOT_RW ;
if (( $? )); then
{
$ECHO "snapshot: could not remount $SNAPSHOT_RW readonly";
exit;
}
fi
At times, we need to monitor the high CPU usage in the system. We can use the below script to monitor the high CPU usage. The script is taken from here.
#!/bin/bash
while true ;do
used=$(free -m |awk 'NR==3 {print $4}')
if [ "$used" -lt 1000 ] && [ "$used" -gt 800 ]; then
echo "Free memory is below 1000MB. Possible memory leak!!!" | /bin/mail -s "HIGH MEMORY ALERT!!!" user@mydomain.com
fi
sleep 5
done
This script allows the root user or admin to add new users to the system in an easier way by just typing the user name and password (The password is entered in an encrypted manner).
#!/bin/bash
# Script to add a user to Linux system
if [ "$(id -u)" -eq 0 ]; then
read -r -p "Enter username : " username
read -r -s -p "Enter password : " password
grep -E "^$username" /etc/passwd >/dev/null
if [ $? -eq 0 ]; then
echo "$username exists!"
exit 1
else
pass=$(perl -e 'print crypt($ARGV[0], "password")' "$password")
useradd -m -p "$pass" "$username"
[ $? -eq 0 ] && echo "User has been added to system!" || echo "Failed to add a user!"
fi
else
echo "Only root may add a user to the system"
exit 2
fi
This script is a pretty basic script useful in backing up the database.
#!/bin/sh
now="$(date +'%d_%m_%Y_%H_%M_%S')"
filename="db_backup_$now".gz
backupfolder="/var/www/vhosts/example.com/httpdocs/backups"
fullpathbackupfile="$backupfolder/$filename"
logfile="$backupfolder/backup_log_"$(date +'%Y_%m').txt
echo "mysqldump started at $(date +'%d-%m-%Y %H:%M:%S')" >> "$logfile"
mysqldump --user=mydbuser--password=mypass --default-character-set=utf8 mydatabase | gzip > "$fullpathbackupfile"
echo "mysqldump finished at $(date +'%d-%m-%Y %H:%M:%S')" >> "$logfile"
chown myuser "$fullpathbackupfile"
chown myuser "$logfile"
echo "file permission changed" >> "$logfile"
find "$backupfolder" -name "db_backup_*" -mtime +8 -exec rm {} \;
echo "old files deleted" >> "$logfile"
echo "operation finished at $(date +'%d-%m-%Y %H:%M:%S')" >> "$logfile"
echo "*****************" >> "$logfile"
exit 0
Do not forget to enable rsync on the synology
This script stop the docker containers, copy the content of /home/pi on the raspberry to a shared folder /backups/raspberry on the synology and restart the containers.
#!/bin/bash
echo Stopping the docker containers at $(date +"%m-%d-%y"-"%T")
# gracefully stopping all the containers
docker stop $(docker ps -q)
wait
echo Running rsync to copy the home dir to the syno
echo Rsync started at $(date +"%m-%d-%y"-"%T")
# swapped -a with -rltDvx to avoid permissions errors since the user "rsync" can't change ownership on the nas
# excluded the files in the cache directory from the backup with --exclude '.cache/*'
# since ssh is not running on port 22 specify a different one with -e 'ssh -p 6712'
rsync -rltDvx --exclude '.cache/*' --stats -e 'ssh -p 6712' /home/pi/ rsync@192.168.1.60:/volume1/backups/raspberry
wait
echo Rsync ended at $(date +"%m-%d-%y"-"%T")
echo Starting the docker containers at $(date +"%m-%d-%y"-"%T")
# restart all the containers
docker restart $(docker ps -a -q)
wait
echo All finished !
Do not forget to make the script executable afterwards ->
chmod u+rx backup.sh
This script backs up the content of /home /var/spool/mail /etc /root /boot /opt to the /home/ds001/backup folder.
Add the task to crontab next.
#!/bin/bash
##########################################
#
# Backup vps directories for DS002 export
#
##########################################
# What to backup.
backup_files="/home /var/spool/mail /etc /root /boot /opt"
# What to exclude.
exclude="/home/ds002/backup"
# Where to backup to.
dest="/home/ds002/backup"
# Create archive filename.
day=$(date +"%m-%d-%y"-"%T")
hostname=$(hostname -s)
archive_file="$hostname-$day.tgz"
# Print start status message.
echo "Backing up $backup_files to $dest/$archive_file"
date
echo
# Backup the files using tar.
tar --exclude=$exclude -zcf $dest/$archive_file $backup_files
# Print end status message.
echo
echo "Backup finished"
date
# Cleaning backups older than 90 days
echo "Cleaning old backups"
find /home/ds002/backup ! -type d -mtime +90 -delete
echo "Finished cleaning"
# Long listing of files in $dest to check file sizes.
ls -lh $dest
This script deletes old files based on a specified number of days
#!/bin/bash
cd /mnt/user/biblio/journaux/
# for files only
# find . -not -path '*/.*' -mtime +7 -type f -delete
# for files and folders
find . -not -path '*/.*' -mtime +7 -type d -exec rm -rf {} \;
ls > liste_journaux.txt
echo "vieux journaux de + de 7 jours effacés !"
cat liste_journaux.txt
The one and only.
The quick and simple editor for cron schedule expressions by Cronitor