Prevent script instances from picking up same task

[freeloader]

Hi guys,

 

For a project I made sort of a custom cron database. Database has 4 columns: ID (auto increment), TaskID, DateTime, Locked.

 

I'm running a 1 minute cron in the form of a php script. The script itself starts with a query that loads a task with 'Locked != 'Y' and DateTime < NOW( ).

 

It then locks the task (by flagging the 'Locked' field in the db) and launches another script that finishes it. That last script deletes the task when finished from the cron database.

 

Problem is, at certain peek hours, the system would get laggy, there'd be a bunch of tasks stacking up and it would get behind on the schedule.

 

In order to combat that, I made an extra 1 minute cron, launching the same script. Now, my problem: mysql is too slow

 

In principle, there shouldn't be any problem: all tasks picked up by either instance of the script would be locked so the other instance wouldn't be able to pick up the same task. The problem occurs when both instances are booted at the same time (well, one after the other but with a minuscule time difference between them) and they both at the same time run the query to get a 'free' task from the database: the system will give them both the same task before either of the script instances has the time to lock it up.

 

I'm trying to think of some solutions but I'd like your feedback on what solution would be best.

 

- Putting an exclusive lock on the php file is not an option for me since I still want to run the script, I just need it to pick up an exclusive task.

- Other option: having the script open with a random sleep of (1, 10) seconds, it will have the script instances pick up a task at a different time, giving the other instance time to lock it up. Obvious disadvantage: I'm losing time.

- Using a file as a flag. Set a directory and create a file in it. Check if this is the only file in the dir, if yes: start right away. Otherwise: go to sleep for 2 seconds (should be plenty of time to run 2 queries in the other instance). What is the fastest method of doing a directory scan though, glob()?

 

My question: what's the fastest/best way to solve this? Thanks!

[batwimp]

You can offset your cron jobs by adding a delay to one of them:

 

00-59 * * * * root sh /home/me/myscript.sh

00-59 * * * * root sleep 30 && sh /home/me/myscript.sh

 

The 'sleep 30' is the offset, 30 seconds. But if you were worried about delaying a job by 1-10 random seconds, this may not be your best choice. I just figured since you were running cron jobs every minute anyway, offsetting one of them wouldn't hurt.

[freeloader]

I was worried about both of them sleeping 10 seconds each. Having only one of them sleep for 5 seconds is optimal. This seems like a great solution, thanks! I'm going to try it out

 

Not sure if my crontab will handle the sleep part (it's a synology NAS, they use a custom sort of crontab).