Friday, April 26, 2019

linux - Filter tail command through multiple grep commands to separate files

What I am trying to do is filter the output of one log file into separated log files based on a grep filter.



tail -f test.log | tee >(grep "Error" > error.log) >(grep "Warning"" >
warning.log)



This way all "Error" entries are in one file, and all "Warning" entries are in a separate file.


I know this works in concept, because if I use cat instead of tail, I get the correct file output, but cannot track changes in real time (which I need because I am watching output logs from actively running tests)


Also, if I remove the '>' file re-directors from the grep commands it outputs the individual grep outputs to the console correctly; however, I want a recorded file as well.


Answer:


When writing to a file the output of grep was being buffered. Using egrep with a --line-buffer option fixed the behavior.


The new command looks like:



tail -f test.log | tee >(egrep --line-buffered "ERROR" > error.log) >(egrep --line-buffered "WARNING" > warning.log)


No comments:

Post a Comment

hard drive - Leaving bad sectors in unformatted partition?

Laptop was acting really weird, and copy and seek times were really slow, so I decided to scan the hard drive surface. I have a couple hundr...