Unix Commands & Scripts

Resolving security alerts by removing an old kernel

While working on an Oracle instance running on a Unix EC2 server, I encountered a recurring security alert indicating that my server was using an outdated kernel version. Despite being up to date with patches, the issue persisted. This blog post outlines the steps I took to address this problem efficiently, allowing me to continue my work without the need for creating a new server.

Identifying the Issue:

To determine the kernel versions installed on my system, I used the command:

rpm -qa | grep kernel

Among the listed versions, I noticed an older kernel, “kernel-uek-5.4.17-2136.319.1.3.el7uek.x86_64,” that was causing the security alert.

Solution Steps:

Switching to Root User:
To make the necessary changes, I switched to the root user using the command:

sudo su root

Removing the Old Kernel:
To remove the outdated kernel version, I utilized the following command:

yum remove kernel-uek-5.4.17-2136.319.1.3.el7uek.x86_64

This command triggered the removal process and displayed the following output:

Loaded plugins: ulninfo
Resolving Dependencies
--> Running transaction check
---> Package kernel-uek.x86_64 0:5.4.17-2136.319.1.3.el7uek will be erased
--> Finished Dependency Resolution

Dependencies Resolved

=====================================================================================================================================================================================================
Package                                     Arch                                    Version                                                       Repository                                   Size
=====================================================================================================================================================================================================
Removing:
kernel-uek                                  x86_64                                  5.4.17-2136.319.1.3.el7uek                                    @ol7_UEKR6                                  115 M

Transaction Summary
=====================================================================================================================================================================================================
Remove  1 Package

Installed size: 115 M
Is this ok [y/N]: y
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Erasing    : kernel-uek-5.4.17-2136.319.1.3.el7uek.x86_64                                                                                                                                      1/1
  Verifying  : kernel-uek-5.4.17-2136.319.1.3.el7uek.x86_64                                                                                                                                      1/1

Removed:
  kernel-uek.x86_64 0:5.4.17-2136.319.1.3.el7uek

Complete!

Executing 7-Zip from the command line and protecting the archive with a password

My past practice to back up my personal files was to copy the files to a couple of USB drives and store them in a fireproof safe. However, with the continued drought and increased risk of fire in the Western States, I was evaluating a cloud-based backup solution. I must confess I am very wary of such a solution as I am placing my data (files) into the hands of a third party.

In my first attempt, I turned on passwords in all the software packages that I use so that the underlying data files will be password protected. However, I continued to feel uneasy about this and wanted another layer of security. I also needed the ability to pick and choose certain files from different directories for the cloud backup. I decide to use 7-Zip as it allowed me to:

a. create a windows shell script that I could run with a mouse click
b. add files from multiple locations
c. protect the encrypted archive with a password

You can download 7-Zip from here

Download 7-Zip

After you install 7-Zip, you can use it via its GUI. I, however, wanted to do this via a shell script. On Windows 10, under the c:\Program Files (x86)\7-Zip directory you will find the executable 7z.exe. You can either add this location to your Windows path variable or directly reference the executable as I have done below.

The commands to add files to an archive are:

"c:\Program Files (x86)\7-Zip\7z.exe" a C:\7zip-archive-location\output-archive.7z "C:\location-01\file-01.txt" -pYour_Super_Secret_Password

"c:\Program Files (x86)\7-Zip\7z.exe" a C:\7zip-archive-location\output-archive.7z "C:\location-02\file-02.txt" -pYour_Super_Secret_Password

"c:\Program Files (x86)\7-Zip\7z.exe" a C:\7zip-archive-location\output-archive.7z "C:\location-03\file-03.txt" -pYour_Super_Secret_Password

The command is broken down as follows:

"c:\Program Files (x86)\7-Zip\7z.exe" 
The location of the executable. Since the location has an embedded                                         space, the entire text is enclosed in double-quotes. For simplicity, I have enclosed all the file locations in double-quotes.

a
Parameter to create (add to) an archive (zip file) 

C:\7zip-archive-location\output-archive.7z
Location and name of the archive  

"C:\location-01\file-01.txt"
The file(s) you want to be included in the archive. Wildcards are permitted.

-pYour_Super_Secret_Password
The password for the file. Note that this parameter is preceded by a hyphen and the password follows the “p” parameter without a space.

After the file is created, I added the below command to list all the files in the archive:

"c:\Program Files (x86)\7-Zip\7z.exe" l C:\7zip-archive-location\output-archive.7z

The command is broken down as follows:

"c:\Program Files (x86)\7-Zip\7z.exe" 
The location of the executable. Since the location has an embedded space, the entire text is enclosed in double-quotes. For simplicity, I have enclosed all the file locations in double-quotes.

l
Parameter to list the files within an archive (zip file) 

C:\7zip-archive-location\output-archive.7z
Location and name of the archive

CRON not executing scripts – Time zone issue

I ran into a strange issue today on a SunOS 5.11 11.4.24.75.2 sun4v sparc sun4v server. When I used the “date” command, I would see the current date and time displayed as (for example) 15:25 ET. I then created a CRON entry for 15:35 ET. However, the script would not execute at the time defined in CRON. I checked the usual permissions etc., but nothing worked. After some trial and error, a colleague identified that the user’s profile had the following entries:

TZ="EST5EDT"
export TZ

The server was actually running GMT, but the time was being displayed in ET. The scripts I was scheduling in the displayed time (ET) were not executing because CRON was scheduling the jobs in GMT.

After the above two lines were commented out, and the CRON entries defined in GMT, the scripts executed at the expected times.

printf wrapping/tabbing on spaces in variables

If you need to print the contents of variables that happen to contain spaces via printf you would typically code as follows:

#!/bin/bash
export var_01='This is variable 01'
export var_02='continuing in variable 02'
printf '%-30s %-30s\n' $var_01 $var_02 >> temp.txt

However, this will output as follows:

This                           is                            
variable                       01                            
continuing                     in                            
variable                       02

Instead of the expected:

This is variable 01            continuing in variable 02

To prevent this, enclose the variables in double quotes:

#!/bin/bash
export var_01='This is variable 01'
export var_02='continuing in variable 02'
printf '%-30s %-30s\n' "$var_01" "$var_02" >> temp.txt

Storing the result from a v$ table into a variable in a script

Typically while selecting from a V$ table in a Unix script, I have to escape the “$” character as shown below:

sqlplus -s $sqlplus_command << EOF > backup_check_temp.txt
  select  value 
  from    V\$RMAN_CONFIGURATION 
  where   NAME = 'RETENTION POLICY'
  ;  
  exit;
EOF

The above will successfully run the SQL and write the output in the spool file.

However, if instead of the output in a spool file, you want the output in a Unix variable for further processing, you would change the code to

export ret_period=`sqlplus -s $sqlplus_command << EOF
                   select   value 
                   from     V\$RMAN_CONFIGURATION 
                   where    NAME = 'RETENTION POLICY'
                   ;
EOF`
if  [ -z "$ret_period" ]; then
    export ret_period='Not found'
fi

This will fail with:

+ export 'ret_period=                select value from V where NAME = '\''RETENTION POLICY'\''
                                       *
ERROR at line 1:
ORA-00942: table or view does not exist'

The reason is that the second method of coding results in two levels of interpretation, each of which requires the “$” character to be escaped. To get the code working, it has to be coded as follows (note the two “\\” escape characters):

export ret_period=`sqlplus -s $sqlplus_command << EOF
                   select   value 
                   from     V\\$RMAN_CONFIGURATION 
                   where    NAME = 'RETENTION POLICY'
                   ;
EOF`
if  [ -z "$ret_period" ]; then
    export ret_period='Not found'
fi

Running an SCP in background while providing a password at the command prompt

I often have the requirement to run an SCP in the background (i.e. not terminally attached) on servers that have not exchanged keys to facilitate password less access. Hence, there is a need to provide a password in the foreground.

To put the SCP into background, enter the SCP command as follows (No “&” at end):

nohup scp -p /source_directory/my_very_large_file.bus myuser@target_server.somedomain.com:/target_directory > nohup.out 2>&1

You will be prompted for the password:

[unix_prompt] nohup scp -p /source_directory/my_very_large_file.bus myuser@target_server.somedomain.com:/target_directory > nohup.out 2>&1
myuser@target_server.somedomain.com's password:

Enter the password and press enter
Then press CTRL Z

Output will look like this. You can see the CTRL Z and a message that states it is stopped:

[unix_prompt] nohup scp -p /source_directory/my_very_large_file.bus myuser@target_server.somedomain.com:/target_directory > nohup.out 2>&1
myuser@target_server.somedomain.com's password: 
^Z
[8]+  Stopped                 nohup scp -p /source_directory/my_very_large_file.bus myuser@target_server.somedomain.com:/target_directory > nohup.out 2>&1

After you see the stopped message and are back at the prompt type in “bg”:

[unix_prompt] bg
[8]+ nohup scp -p /source_directory/my_very_large_file.bus myuser@target_server.somedomain.com:/target_directory > nohup.out 2>&1 &
[unix_prompt]

The SCP will now be resumed in background:

[unix_prompt] ps -ef | grep myuser@target_server
oracle   42065 54077  1 18:16 pts/2    00:00:02 scp -p /source_directory/my_very_large_file.bus myuser@target_server.somedomain.com:/target_directory
oracle   55618 54077  0 18:18 pts/2    00:00:00 grep myuser@target_server

The SCP is now running in the background and the terminal is freed.

Cleaning up Oracle audit files resulting in “cannot execute [Argument list too long]”

Just a quick little Unix tip:

Over the Christmas/New Year’s holidays one of my application’s audit file destination started filling up causing the database to stop future logons and generate page outs etc. The root cause appeared to be a script running on an application server that for some unknown reason was attempting to logon and failing about 2000 times a minute. This generated about the same number of audit files filling the mount point and/or exceeding the inodes. The regular housekeeping script was unable to clean up the files as it was executing the command similar to:

rm *_ora_3*_20181231*.aud

This would result in the error:

-ksh: rm: /bin/rm: cannot execute [Argument list too long]

This being the holiday season, we were unable to find a contact to either stop or fix the application script. After some Google searches, I was able to rapidly delete the files with the below command:

find /app/oracle/admin/instance1/adump -maxdepth 1 -name "*.aud" -print0 | xargs -0 rm

Unix send a mail with body and attachment

If you need to send an email from the command line or a script that contains both a mail body and an attachment you can use the below command:

mailx -s "Subject of this email " -r "sent-from@somewhere.com" -a file_name.txt sent-to@somewhere.com << EOF
This will be the first line of the body of the email.
2nd line.
3rd line.
EOF

Odd listing of file when using an environment variable

I had a file similar to the one shown below

$ more dean.txt
2017-12-02 18:16:57 17279 [ERROR] ** TEST ERROR ** Error_code: 1007

When I ran the below command from the command line, it displays the correct information:

$ cat dean.txt | grep -i error_code                              
2017-12-02 18:16:57 17279 [ERROR] ** TEST ERROR ** Error_code: 1007

However, when I set up an environment variable as:

$ DEAN=`cat dean.txt | grep -i error_code`

And I run the command via the environment variable:

$ echo $DEAN                                                     

2017-12-02 18:16:57 17279 [ERROR] 0 bin COPYING COPYING.AGPLv3 COPYING.GPLv2 COPYING-jemalloc dean.txt docs include lib log_archives man my.cnf my.cnf.bak.20171121 my.cnf.dean.bak mysql-files mysql-test PATENTS README.md README.MySQL share support-files TEST ERROR 0 bin COPYING COPYING.AGPLv3 COPYING.GPLv2 COPYING-jemalloc dean.txt docs include lib log_archives man my.cnf my.cnf.bak.20171121 my.cnf.dean.bak mysql-files mysql-test PATENTS README.md README.MySQL share support-files Error_code: 1007

It lists all the files in the directory because it is interpreting the “*” in the file’s contents as a listing of all files.

Change the file contents to:

$ more dean.txt
2017-12-02 18:16:57 17279 [ERROR]  TEST ERROR  Error_code: 1007

And reset the environment variable:

$ DEAN=`cat dean.txt | grep -i error_code`

And execute the command

$ echo $DEAN                              
2017-12-02 18:16:57 17279 [ERROR] TEST ERROR Error_code: 1007

The workaround is to put the variable in quotes as shown below.

$ more dean.txt
2017-12-02 18:16:57 17279 [ERROR] ** TEST ERROR ** Error_code: 1007

$ DEAN=`cat dean.txt | grep -i error_code`                       

$ echo "$DEAN"
2017-12-02 18:16:57 17279 [ERROR] ** TEST ERROR ** Error_code: 1007

scp error – no matching cipher found

I was performing an scp from an older server to a newer server with the command:

scp local_file remote_user@remote_server.something.com:/remote_directory

and got an error message:

no matching cipher found: client 3des-cbc,blowfish-cbc,cast128-cbc server aes128-ctr,aes192-ctr,aes256-ctr,aes256-cbc
lost connection

I was ble to get around this by explicitly providing the cipher:

scp -o Ciphers=aes256-ctr local_file remote_user@remote_server.something.com:/remote_directory