Wednesday, November 23, 2016

linux - Pre-pending to an output line of `tail -f file` if its contents match another file



Trying to cram all of the following into the title was hard, so this is what I really mean:



I am doing a tail -F access.log to watch an Apache2 log in realtime. The first thing on each line is an IP address followed by a space. I also have an unchanging CSV file where each row is an IP address, comma, name. For example:




10.1.2.33,John Smith
10.1.2.190,Jane Doe



Deliverable: I want to follow my access log as before, but if the CSV happens to contain a row with the same IP address that the current logfile line starts with, then insert the person's name at the start of the line.



For example, a line like this:



10.1.2.33 - - [22/Aug/2013:13:41:24 +0000] "GET /index.php ...



Should instead render as:




John Smith 10.1.2.33 - - [22/Aug/2013:13:41:24 +0000] "GET /index.php ...



Ideally, I would like to do this entirely by piping some regex commands together. So far, my biggest issue is that grep doesn't accept a pattern on stdin, it expects the subject on stdin (unless I'm missing something). Also, I have no problem transforming the CSV into some better format if required. Thanks.


Answer



Here's a POSIX shell solution



tail -f access.log | while read -r line;do
ip=${line%% *}
name=$(grep -F "$ip" your_csv_file|cut -d, -f2)
if [ -z "$name" ];then

printf "%s\n" "$line"
else
printf "%s\n" "$name $line"
fi
done


Edit



Two improvements made to the solution:





  • -r switch added to read so that it doesn't evaluate escape sequences in the lines it reads

  • -F switch added to grep so that it treats the IP as a fixed string rather than a regular expression.


No comments:

Post a Comment

hard drive - Leaving bad sectors in unformatted partition?

Laptop was acting really weird, and copy and seek times were really slow, so I decided to scan the hard drive surface. I have a couple hundr...