linux - parse log file with heavy output

I'd like to monitor my vsftpd logs and pull information from the logs to process the files that have been uploaded. Up till now everything was great and worked fine:

tail -n0 -F /var/log/vsftpd.log | while read line; do

    if echo "$line" | grep -q 'OK UPLOAD:'; then
        #do a bunch of processing stuff - this takes some time
    fi
done

I started doing some scale testing and uploaded 500 files at once. For some reason the log lines are missing or they are cut off when you upload so many things. For example a typical line looks like:

Sun Apr 7 09:08:51 2013 [pid 25409] [cam02430] OK UPLOAD: Client "206.132.183.201", "/20130407/09/20130407_090842D.avi", 531792 bytes, 426.14Kbyte/sec

But sometimes the line looks like this (if not missing entirely):

:08:51 2013 [pid 25409] [cam02430] OK UPLOAD: Client "206.132.183.201", "/20130407/09/20130407_090842D.avi", 531792 bytes, 426.14Kbyte/sec

It is cutting off the first few characters of the log. It doesn't do it for every line but most. Also many lines are just missing. I guess I have a race condition of my processing not finishing in time. How I can get around this?

EDIT
I guess I have 3 options:

1. Don't use tail and try to use a combination of things like a cron job/grep to check the log for a new entry. I imagine this could be tricky. How do I tell what is new in that log?




2. Monitor the file change with monit or equivalent. I don't believe this is an option. The files get stored into somewhat random directories with files names that are time stamped.




3. logstash or equivalent. I don't think these programs fit my needs. But maybe someone knows otherwise.

Right now I'm focusing on #1 with no good leads. So any thoughts on any of this would be appreciated.