Month: February 2017

Processing a text file with spaces in the data and a file separator

I needed to code a script that would read through a text file and process some data. The format of the file was as follows:

Item Id|Item Name
01|Car
02|Bus
03|Plane

The code was:

#!/usr/bin/ksh
for parm_line in `cat dean_test.txt`
do
    export item_id=$(echo $parm_line | cut -f1 -d\|)
    export item_name=$(echo $parm_line | cut -f2 -d\|)
 
    if  [ $item_id != "Item Id" ]; then
        echo $parm_line
    fi
done

My expectation was that it would skip the header line and process the three data lines. The output looked like:

Item
Id|Item
Name
01|Car
02|Bus
03|Plane

This was being caused by the fact that the header row had spaces in the names of the fields. I had to change the code as follows and add the IFS:

#!/usr/bin/ksh
 
IFS=$'\n' 
 
for parm_line in `cat dean_test.txt`
do
    export item_id=$(echo $parm_line | cut -f1 -d\|)
    export item_name=$(echo $parm_line | cut -f2 -d\|)
 
    if  [ $item_id != "Item Id" ]; then
        echo $parm_line
    fi
done

This change fixed the issues and I got the correct output:

01|Car
02|Bus
03|Plane

The short snippet of code was part of a much larger script. I could not use a WHILE loop as I was going to be issuing ssh commands.

Read tar.gz files without extracting

I had a need to look for certain commands in a list of Oracle audit logs what had been tarred and zipped (*.tar.gz format). I created a list with an ls command:

ls *.gz > dean.txt

I then used the below python code to read through the zipped tar balls looking for certain strings:

#!/usr/bin/python
import tarfile,os
import sys
 
list_of_tar = open('dean.txt', 'r')
for tar_name in list_of_tar:
    tar_name = tar_name.rstrip()
    print tar_name
    tar = tarfile.open(tar_name,'r:gz')
    for member in tar.getnames():
        file_name=tar.extractfile(member)
        for line in file_name:
            line = line.lower()
            if 'alter ' in line:
                if ' system ' in line:
                    if 'kill' in line:
                        print tar_name, member , line
tar.close()

This was a quick and dirty python to fulfill an immediate need. With more time the search condition could be improved.

“No space left on device” despite being less than 100% used

We were encountering issues writing to a mount point on a server: 

[test-server] ls > dean
-bash: dean: No space left on device

The file system was 18% free:

[test-server] df -h
Filesystem            Size  Used Avail Use% Mounted on
/file-system           99G   77G   18G  82% /mount-point

The issue was caused by the inodes being used up:

[test-server] df -i /mount-point
Filesystem            Inodes   IUsed IFree IUse% Mounted on
/file-system         6553600 6553600     0  100% /mount-point

Working on the assumption that the largest directory would have the largest number of files

du -h /mount=point | grep '[0-9]G' | sort -nr -k1

I identified the mount point and cleaned up the files.

Hail mary on db links

We had a situation wherein a colleague was attempting to get DB Links working. This was on a 11.2.0.4 database where password obfuscation is not allowed and the users were unsure of the passwords. After multiple changes to the passwords on the target system and drops and recreates of the links, the users continued to encounter errors while using the db link. We looked up a number of posts by different people on the web and then tried

alter system flush shared pool;

This resolved the issue. This was a non production environment. Please consider all the implications before you issue a flush in a production environment.