I often see a log where I want to print sequential columns in the log for a purpose.

Problem

How to print all columns from the nth to the last?

Solution

For example, you have a test.txt file that contains the following:

No  Name        Address
1   Richard     Apt. 344 86094 Swaniawski Drive, East Suzetteshire, MT 51323-2013
2   Alex        4522 Rosenbaum Island, Lake Suzan, IL 68193
3   Bryan       Apt. 907 703 Douglas Run, West Brainburgh, MT 70080-8990

Usually, to display the complete Address column, I will run the command below:

awk '{print $3,$4,$5,$6,$7,$8,$9,$10,$11,$12}' test.txt 

However, I think this method is less effective because if the address is more than 12 columns, then I have to write more than 12 items, and it is very tiring. After searching on the internet, there are 2 methods you can use:

1. Using the awk command

From the test.txt file, you just want to print the entire column except columns 1 and 2, then you can use the command below:

awk '{$1=$2=""; print $0}' test.txt 

The result will be as shown in the image below:

2. Using the cut command

In addition to using awk, you can also use the cut command to display the same by using the command below:

cut -d' ' -f3- test.txt 

And the result will be as shown in the image below when you run the command above:

cut -d" " -f7- test.txt

Note

There are still several methods to print columns that run from the nth to the last column, but I think those two methods will suffice.

References

testingbot.com
stackoverflow.com

I have a file containing many IPs that I want to ping the rest of the IPs.

Problem

How to read a file line by line on Linux?

Solution

I have an ip.txt file that contains IP addresses, and in this article, I will only limit it to 3 IP addresses:

192.168.56.2
192.168.56.12
192.168.56.100

To read a file line by line on Linux, you can use the format below:

cat your_file | while read line
do
         the-commands-that-you-want-to-run
         .......
         .......
done

In this case, the format above has changed as below to ping each IP in the file:

cat ip.txt | while read line
do
	ping -c 3 $line
	echo
done

The command above means to run the ping command 3x on each IP in the ip.txt file by separating one line for each IP. If we run the command, it will look like the image below:

Change the script above to be as below if you want to enter the results of each ping IP into one file:

cat ip.txt | while read line
do
    ping -c 3 $line
    echo
done > ping_ip.txt

Look at the picture below:

But if you want to enter the results of each ping IP into several files based on the IP, change the script above to be as below:

cat ip.txt | while read line
do
    ping -c 3 $line
    echo
done > ping_ip.txt

There should be new files after you run the script above, and each file will contain the results of the ping IP as in the image below:

Note

In addition to using the script above, you can use the script below to read line by line in a Linux file:

while IFS= read -r line; do ping -c 3 $line ; echo; done < ip.txt

References

cyberciti.biz
linuxhint.com
linuxize.com
stackoverflow.com

I need to get a value between 2 special characters I use for my other purposes in the log file on the Linux server.

Problem

How to get the value between two special characters on Linux?

Solution

Special characters are the punctuation characters on your keyboard, such as !, @, #, and so on. After I searched on the internet, there were 2 solutions to get the value between these two special characters: using the grep command or using the cut command.

1. Using the grep command

To get the value between two special characters using the grep command, use the following format:

grep -Po "(?<=\special_character).*?(?=\character_special)"

For example, if the special character is brackets or (), then the format above changes to:

grep -Po "(?<=\().*?(?=\()"

Type the command below to get the value between the brackets:

echo "(test)" | grep -Po "(?<=\().*?(?=\))"

Look at the example in the image below:

2. Using the cut command

To get the value between two special characters using the grep command, use the following format:

cut -d "special_character" -f2 | cut -d "special_character" -f1

For example, if the special character is pound or #, then the format above changes to:

cut -d "#" -f2 | cut -d "#" -f1

Type the command below to get the value between the pounds:

echo "#test#" | cut -d "#" -f2 | cut -d "#" -f1

Look at the example in the image below:

Note

If you have a log file in Linux, for example, the content of the file is as image below:

nginx1 server status is [OK]
nginx1 server status is [OK]
db1 server status is [OK]
db2 server status is [OK]
redis1 server status is [OK]
redis2 server status is [OK]
monitoring server status is [OK]

You can use one of the commands above to get the value between 2 special characters, and I use the cut command as in the command below:

cat test.txt | cut -d "[" -f2 | cut -d "]" -f1

References

ers.texas.gov
stackoverflow.com

The previous article explained how the Molly-Guard tool can protect Linux servers from accidental reboot or shutdown commands. Unfortunately, this tool is only available on Debian/Ubuntu distros and their derivatives, while sysadmins generally have many Linux servers from various distros.

Problem

How to make a tool like Molly-Guard using a bash script?

Solution

To create a tool like Molly-Guard, you can use a bash script, and this script has been tested on Ubuntu Server 24.04, RockyLinux9, and OpenSUSE 15 distros. And this script should be applied throughout the Linux distro to replace the Molly Guard tool. Here are the steps:

1. Check the paths

First, check where the reboot, shutdown, poweroff, and halt commands are located on the Linux server by running the command below:

whereis -b reboot
whereis -b shutdown
whereis -b poweroff
whereis -b halt

As far as I know, Linux distributions such as RockyLinux and Ubuntu only provide one binary file for each command for the reboot, shutdown, poweroff, or halt command, usually in the folder /usr /sbin. However, on certain distros, for example, the OpenSUSE distro, when you run the command above, the result will be as shown in the image below:

From the image above, you can see that 2 files represent each of these commands. Because this article uses the commands in the /usr/sbin folder, you can rename the commands in the /sbin folder using the commands below:

mv /sbin/reboot /sbin/reboot_backup 
mv /sbin/shutdown /sbin/shutdown_backup
mv /sbin/poweroff /sbin/poweroff_backup
mv /sbin/halt /sbin/halt_backup

2. Create a bash script

Copy the bash script below into the /usr/local/bin/ folder and give it a molly-guard-costume.sh name:

#!/usr/bin/env bash

# molly-guard-custome.sh: Prevent accidental reboot or shutdown like molly-guard tool

##########################################################################################
# The functions

check_hostname_reboot() {
# Compare the user input with the actual hostname
if [ "$USER_INPUT" == "$ACTUAL_HOSTNAME" ]; then
        echo "Hostname confirmed."
        echo
        sleep 1
        echo "Proceeding with the command..."
        echo
        sleep 1
        echo "The system will reboot now!"
        echo
        sleep 1
        /usr/sbin/reboot_server 
else
        echo "Hostname mismatch. Aborting the reboot operation."
        exit 1
fi
}

check_hostname_halt() {
# Compare the user input with the actual hostname
if [ "$USER_INPUT" == "$ACTUAL_HOSTNAME" ]; then
        echo "Hostname confirmed."
        echo
        sleep 1
        echo "Proceeding with the command..."
        echo
        sleep 1
        /usr/sbin/halt_server 
else
        echo "Hostname mismatch. Aborting the halt operation."
        exit 1
fi
}

check_hostname_poweroff() {
# Compare the user input with the actual hostname
if [ "$USER_INPUT" == "$ACTUAL_HOSTNAME" ]; then
        echo "Hostname confirmed."
        echo
        sleep 1
        echo "Proceeding with the command..."
        echo
        sleep 1
        echo "The system will reboot now!"
        echo
        sleep 1
        /usr/sbin/poweroff_server 
else
        echo "Hostname mismatch. Aborting the poweroff operation."
        exit 1
fi
}

check_hostname_shutdown() {
# Compare the user input with the actual hostname
if [ "$USER_INPUT" == "$ACTUAL_HOSTNAME" ]; then
        echo "Hostname confirmed."
        echo
        sleep 1
        echo "Proceeding with the command..."
        echo
        sleep 1
        echo "The system will reboot now!"
        echo
        sleep 1
        /usr/sbin/shutdown_server 
else
        echo "Hostname mismatch. Aborting the shutdown operation."
        exit 1
fi
}

##########################################################################################

# Get the actual system hostname
ACTUAL_HOSTNAME=$(hostname)

# Ask the user to input the hostname
echo "Please confirm the hostname to proceed it."
read -p "Enter the hostname: " USER_INPUT

# Check the command
ps aux | grep reboot > /tmp/reboot.txt
ps aux | grep halt > /tmp/halt.txt
ps aux | grep poweroff > /tmp/poweroff.txt
ps aux | grep shutdown > /tmp/shutdown.txt

reboot_size=`ls -al /tmp/reboot.txt | awk '{print $5}'`
halt_size=`ls -al /tmp/halt.txt | awk '{print $5}'`
poweroff_size=`ls -al /tmp/poweroff.txt | awk '{print $5}'`
shutdown_size=`ls -al /tmp/shutdown.txt | awk '{print $5}'`

# Compare the command

if [ $reboot_size -gt 90 ];
then
        rm -f /tmp/reboot.txt /tmp/halt.txt /tmp/poweroff.txt /tmp/shutdown.txt
        check_hostname_reboot
elif [ $shutdown_size -gt 90 ];
then
        rm -f /tmp/reboot.txt /tmp/halt.txt /tmp/poweroff.txt /tmp/shutdown.txt
        check_hostname_shutdown
elif [ $poweroff_size -gt 90 ];
then
        rm -f /tmp/reboot.txt /tmp/halt.txt /tmp/poweroff.txt /tmp/shutdown.txt
        check_hostname_poweroff
elif [ $halt_size -gt 90 ];
then
        rm -f /tmp/reboot.txt /tmp/halt.txt /tmp/poweroff.txt /tmp/shutdown.txt
        check_hostname_halt
fi

Then run the command below so that it can be run:

chmod +x /usr/local/bin/molly-guard-costume.sh

3. Copy the commands

Use the commands below to copy the commands :

file_path=$(whereis -b "reboot" | cut -d ' ' -f 2);sudo cp $file_path ${file_path}_server > /dev/null 2>&1
file_path=$(whereis -b "shutdown" | cut -d ' ' -f 2);sudo cp $file_path ${file_path}_server > /dev/null 2>&1
file_path=$(whereis -b "poweroff" | cut -d ' ' -f 2);sudo cp $file_path ${file_path}_server > /dev/null 2>&1
file_path=$(whereis -b "halt" | cut -d ' ' -f 2);sudo cp $file_path ${file_path}_server > /dev/null 2>&1

To see the result, type the command below:

ls -al /usr/sbin/ | grep server

4. Link the script to the commands

Then, link the bash script to the commands by running the command below:

ln -sf /usr/local/bin/molly-guard-costume.sh /usr/sbin/reboot
ln -sf /usr/local/bin/molly-guard-costume.sh /usr/sbin/shutdown
ln -sf /usr/local/bin/molly-guard-costume.sh /usr/sbin/poweroff
ln -sf /usr/local/bin/molly-guard-costume.sh /usr/sbin/halt

To see the results, type the command below:

ls -al /usr/sbin/ | grep molly

5. Test the result

Now, try to do the reboot command and write the wrong hostname, and your Linux server shouldn’t reboot. However, try to write the correct hostname, then your Linux server should be rebooted, like in the image below:

You should get the same results when running other commands such as poweroff, shutdown, and halt.

Note

Unlike the Molly-Guard tool, this script will continue to work even though you run the reboot or shutdown command without an SSH connection or directly connect the keyboard to the Linux server.

References

unix.bris.ac.uk
stackoverflow.com
geeksforgeeks.org

The previous article explained how to see the status of a public IP, whether it is indicated as spam or not, using a PHP script. This article will explain the status of a public IP that is indicated as spam or does not use bash scripts.

Problem

How to check a public IP in the spam list using a bash script?

Solution

To run the bash script to check whether a public IP address in the spam list is spam or not, you must install the required packages below:

Ubuntu/Debian

apt-get install -y dnsutils

RHEL/CentOS/RockyLinux/AlmaLinux

yum install bind-utils -y

Then copy the bash script below and give the name check_ip_spam.sh:

#!/usr/bin/env bash
# -- $Id: blcheck,v 1.4 2007/06/16 01:08:10 j65nko Exp $ --
# Check if an IP address is listed on one of the following blacklists
# The format is chosen to make it easy to add or delete
# The shell will strip multiple whitespace
BLISTS="
bl.spamcop.net
cbl.abuseat.org
dnsbl.justspam.org
dnsbl.sorbs.net
relays.mail-abuse.org
spam.dnsbl.sorbs.net
spamguard.leadmon.net
zen.spamhaus.org
"

# simple shell function to show an error message and exit
# $0 : the name of shell script, $1 is the string passed as argument
# >&2 : redirect/send the message to stderr
ERROR() {
echo $0 ERROR: $1 >&2
exit 2
}

# -- Sanity check on parameters
[ $# -ne 1 ] && ERROR 'Please specify a single IP address'

# -- if the address consists of 4 groups of minimal 1, maximal digits, separated by '.'
# -- reverse the order
# -- if the address does not match these criteria the variable 'reverse will be empty'
reverse=$(echo $1 |sed -ne "s~^\([0-9]\{1,3\}\)\.\([0-9]\{1,3\}\)\.\([0-9]\{1,3\}\)\.\([0-9]\{1,3\}\)$~\4.\3.\2.\1~p")
if [ "x${reverse}" = "x" ] ; then
ERROR "IMHO '$1' doesn't look like a valid IP address"
exit 1
fi

# Assuming an IP address of 11.22.33.44 as parameter or argument
# If the IP address in $0 passes our crude regular expression check,
# the variable ${reverse} will contain 44.33.22.11
# In this case the test will be:
# [ "x44.33.22.11" = "x" ]
# This test will fail and the program will continue
# An empty '${reverse}' means that shell argument $1 doesn't pass our simple IP address check
# In that case the test will be:
# [ "x" = "x" ]
# This evaluates to true, so the script will call the ERROR function and quit
# -- do a reverse ( address -> name) DNS lookup
REVERSE_DNS=$(dig +short -x $1)
echo IP $1 NAME ${REVERSE_DNS:----}
EXITCODE=0

# -- cycle through all the blacklists
for BL in ${BLISTS} ; do

# print the UTC date (withour linefeed)
printf $(env TZ=UTC date "+%Y-%m-%d_%H:%M:%S_%Z")

# show the reversed IP and append the name of the blacklist
printf "%-40s" " ${reverse}.${BL}."

# use dig to lookup the name in the blacklist
#echo "$(dig +short -t a ${reverse}.${BL}. | tr '\n' ' ')"
LISTED="$(dig +short -t a ${reverse}.${BL}.)"
echo [${LISTED:-OK}]
echo $LISTED | grep '127\.' >/dev/null && EXITCODE=4
done
exit $EXITCODE
# --- EOT ------

Type the command below so that the bash script can run:

chmod +x check_ip_spam.sh

To run this bash script, use the format below:

./check_ip.sh public_IP_address

For example, you want to check IP 172.217.194.113, then run the script by:

./check_ip.sh 172.217.194.113

And there will be the following display:

From the image above, it can be seen that the public IP does not include spam. If a public IP is included in the spam list, for example, IP 24.209.96.220, it will come out [127.0.0.x] as in the image below:

If you want to check over one IP, then use the syntax format:

for X in public_ip_address_1 public_ip_address_2 ...; do echo;./check_ip $X; echo; done

For example, if you want to check two public IP addresses, 172.217.194.113 and 24.209.96.220, you can type:

for X in 172.217.194.113 24.209.96.220 ; do echo; ./check_ip.sh $X ;echo; done

Note

If you want to change the DNSBL or Domain Name System Blacklists list, then you can change it in lines 7-14 of the scrip,t and you can add the DNSBL list here. The more you enter the DNSBL list, the more valid the output will be.

References

daemonforums.org
maxmind.com
cyberciti.biz
tecmint.com