Retriggerable timer

[PhilFreak]

I'm writing PHP in a web server environment behind nginx. When we receive a page request I want to issue a reply quickly and then start a timer. When the timer has expired I want to do some more work. Experiments with flush() were unfruitful so I split the work into two files and put this at the end of the first file:

shell_exec("nohup php second.php $parameter 1> second.log 2> second.err");

The second file starts with a sleep() with the timeout set at around 60 seconds.

It works but I really want to improve it by allowing the timer to be retriggered if another page request comes in so that second.php only runs once after all the page requests have been handled.

Any ideas?

[gizmola]

What is the application or problem you are trying to solve?  This has all the hallmarks of an X/Y problem.   What I can deduce:

 

•  Some event occurs and some action is taken
• A 2nd action should be taken some time later (60 seconds in your case)
  • However you don't want action 2 to occur in some circumstances for reasons undisclosed

You've used the phrase:  

Quote

It works but I really want to improve it by allowing the timer to be retriggered if another page request comes in so that second.php only runs once after all the page requests have been handled.

 

From your description, every request is immediately handled.  Without knowing the purpose of this 2nd action, it's difficult to provide advice, but the obvious problem is that you want the 2nd action to be aware of the first action.   Rather than a dumb process blocking for 60 seconds, it appears you want a process that

• will be created with a future event datetiime (1 minute in the future)
• If before it completes, a new event #1 comes in, you modify the expiration datetime and set it to 1 minute from the event

 

Another possible low tech way of handling this would be to have process 2 implement a semaphore/lock file that is checked for when process 2 is run. 

Shared memory and IPC semaphores can be helpful for something like this.  With that said, anytime you utilize a mechanism that relies on a single server architecture the scheme is inherently non scalable.

This is where things like queues or databases typically come into play.  Using some in memory server like redis is often a better platform.

[PhilFreak]

Since you ask, the second process is to analyse the page requests for malicious activity and, if necessary, blacklist and report the remote ip.

[gizmola]

1 hour ago, PhilFreak said:

Since you ask, the second process is to analyse the page requests for malicious activity and, if necessary, blacklist and report the remote ip.

But what is the 1st task, and how is it connected to this?

My kneejerk reaction is that there are FOSS IDS tools like OSSEC you should look into.

Even if you continue to go forward, an asynchronous approach is going to be better.   When your site is inevitably accessed by bots/spiders, the overhead of spawnng a php process for every request is likely one that you will regret.

 

 

 

[PhilFreak]

Which is precisely why I'm trying to aggregate requests.

I'd have thought that this would be such a frequent requirement that I'm surprised there isn't a ready-made solution.

[requinix]

You're right, it is a frequent requirement. It's so frequent that people don't implement these things themselves.

If you want to monitor for "bad" activity and ban clients, that's literally what tools like fail2ban are made for.

[gizmola]

5 hours ago, PhilFreak said:

Which is precisely why I'm trying to aggregate requests.

I'd have thought that this would be such a frequent requirement that I'm surprised there isn't a ready-made solution.

Not according to what you originally stated.  You stated that for every request you wanted to "issue a reply quickly and start a timer."  It was never clear if this was just a means to an end or not, because you didn't explain the problem you are trying to solve.

What it does sound like at this point, is that you are trying to create your own home grown IDS or WAF, and you already got a suggestion from me, and a suggestion from requinix.  For the most part people use fail2ban to drop annoying ssh bots and other similar port based traffic by bots and script kiddies trying brute force password attacks.  It's written in Python, so it's not exactly light weight either, but it also has a simpler job in practice -- just count a small number of bad attempts and block the IP.  That isn't going to work for something more sophisticated.

This is why I suggested looking at OSSEC, and if it's more a WAF you want there are bunch of self hosted ones that also have FOSS versions like Safeline, Modsecurity and Bunkerweb.