Jump to content

Locking a file for passing data


NotionCommotion
 Share

Go to solution Solved by kicken,

Recommended Posts

For testing only, I wish to use a file as a simple queue.  Yea, I know I should be using a real queue, but I also wish to learn how to do it, and it is just for testing purposes only.  Is this correct?  Throughput will be low.  Thanks

 

Add to the buffer.

    public function saveTestData($json, $from, $to, $error=false)
    {
        $file='/var/www/test/buffer.json';
        $fp = fopen($file, 'r+');
        while (!flock($fp, LOCK_SH)) { usleep(1); }
        $existing = fread($fp, filesize($file));
        $new=$existing?json_decode($existing,true):[];
        $new[]=['from'=>$from,'to'=>$to, 'message'=>json_encode($json),'error'=>$error];
        ftruncate($fp, 0);
        fwrite($fp, json_encode($new));
        fflush($fp);
        flock($fp, LOCK_UN);
        fclose($fp);
    }

 

Get the buffer data and delete it.

function getTestData()
{
    $file='/var/www/test/buffer.json';
    $fp = fopen($file, 'r+');
    while (!flock($fp, LOCK_SH)) { usleep(1); }
    $existing = fread($fp, filesize($file));
    ftruncate($fp, 0);
    fwrite($fp, '');
    fflush($fp);
    flock($fp, LOCK_UN);
    fclose($fp);
    return $existing;
}
Edited by NotionCommotion
Link to comment
Share on other sites

The way you use flock() generally makes no sense. Your while loop indicates you expect non-blocking behavior, but you haven't set the LOCK_NB flag, so the function is blocking. What do you want? If you switch to non-blocking behavior and simply wait and try, you can end up waiting forever.

Link to comment
Share on other sites

That wasn't the question.

 

flock() is blocking by default: If the file is already locked, then the process is suspended, added to a list and automatically woken up when the file is ready. This means there is no need for any waiting logic. The alternative is non-blocking behavior where the function immediately returns in any case. This implies you do have to write your own waiting logic, and if you get this wrong, a process may lose against other processes over and over again.

 

Since you appearently don't know what you want, I strongly recommend the default blocking behavior. So get rid of the while loop.

Link to comment
Share on other sites

What confused me was this example from http://php.net/manual/en/function.flock.php.  If flock is blocking, what is the purpose of the if statement?

 

So, my writeSomething() will do the following?

  1. Open a file.
  2. Wait until it is available and instantly place a lock on it.
  3. Truncate all data to zero (i.e. delete all content).
  4. Add $txt to a buffer.
  5. Flush it (i.e. write the buffer to the file).
  6. Release the lock.
  7. Close the file.
<?php

function writeSomething($txt) {
    #Example #1 flock() example
    $fp = fopen("/tmp/lock.txt", "r+");

    if (flock($fp, LOCK_EX)) {  // acquire an exclusive lock
        ftruncate($fp, 0);      // truncate file
        fwrite($fp, $txt);
        fflush($fp);            // flush output before releasing the lock
        flock($fp, LOCK_UN);    // release the lock
    } else {
        echo "Couldn't get the lock!";
    }

    fclose($fp);
}

writeSomething('hello');
Link to comment
Share on other sites

Still questioning why if flock is blocking, why does the PHP docs use the IF statement.

 

This is what I now have.  Without even using the getTestData function, for some reason I am not appending the new records, but writing over the buffer.

 

function saveTestData($msg, $from=null, $to="null", $error=false) {
    $file='/var/www/test/buffer.json';
    $fp = fopen($file, "r+");
    if (flock($fp, LOCK_EX)) {
        $records=($filesize=filesize($file))?json_decode(fread($fp, $filesize),true):[];
        $records[]=['from'=>$from,'to'=>$to, 'message'=>json_encode($msg),'error'=>$error];
        ftruncate($fp, 0);
        fwrite($fp, json_encode($records));
        fflush($fp);
        flock($fp, LOCK_UN);
    } else {
        echo "Couldn't get the lock!";
    }
    fclose($fp);
}


function getTestData()
{
    $file='/var/www/test/buffer.json';
    $fp = fopen($file, "r+");
    if (flock($fp, LOCK_EX)) {
        $records=($filesize=filesize($file))?fread($fp, $filesize):'[]';
        ftruncate($fp, 0);
        fwrite($fp, '');
        fflush($fp);
        flock($fp, LOCK_UN);
    } else {
        echo "Couldn't get the lock!";
    }
    fclose($fp);
    return $records;
}
Edited by NotionCommotion
Link to comment
Share on other sites

Still questioning why if flock is blocking, why does the PHP docs use the IF statement.

Because it's possible that flock fails for reasons other than the file already being locked. The way the default blocking behavior essentially works is it will only keep retrying as long as the error returned indicates that the file is already locked. If some other error occurs it'll just fail entirely.

 

The PHP manual doesn't really expand on all the possible errors. If you look at the underlying flock or fcntl function you can get an idea of what other kinds of situations might cause it to fail.

 

EBADF fd is not an open file descriptor

 

EBADF cmd is F_SETLK or F_SETLKW and the file descriptor open mode doesn't match with the type of lock requested.

 

EDEADLK It was detected that the specified F_SETLKW command would cause a deadlock.

 

EFAULT lock is outside your accessible address space.

 

EINTR cmd is F_SETLKW or F_OFD_SETLKW and the operation was interrupted by a signal; see signal(7).

 

EINTR cmd is F_GETLK, F_SETLK, F_OFD_GETLK, or F_OFD_SETLK, and the operation was interrupted by a signal before the lock was checked or acquired. Most likely when locking a remote file (e.g., locking over NFS), but can

sometimes happen locally.

 

ENOLCK Too many segment locks open, lock table is full, or a remote locking protocol failed (e.g., locking over NFS).

Edited by kicken
  • Like 1
Link to comment
Share on other sites

UGGG.  What am I doing wrong?  As seen, it only saves the last entry. Furthermore, if I run the file a second time, additional ^@^@^@^@ content is added (cat doesn't display it, but vi does and the content is obviously no longer valid json).
 
Thanks!
<?php
$file='myjson.json';

addRecord($file,'hello1');
addRecord($file,'hello2');
addRecord($file,'hello3');

echo(file_get_contents($file));

function addRecord($file,$msg) {
    $fp = fopen($file, "r+");
    if (flock($fp, LOCK_EX)) {
        $filesize=filesize($file);
        $json=$filesize?fread($fp, $filesize):'[]';
        $arr=json_decode($json);
        $newRow=(object)['name'=>$msg];
        $arr[]=$newRow;
        $new_json=json_encode($arr);
        ftruncate($fp, 0);
        fwrite($fp, $new_json);
        fflush($fp);
        flock($fp, LOCK_UN);
    }
    else{
        exit('Could not lock the file!!!');
    }
    fclose($fp);
}
Browser Output:
[{"name":"hello3"}]
[Michael@devserver public]$ vi  myjson.json
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@[{"name":"hello3"}]

 

Link to comment
Share on other sites

Thanks Jacques1,

 

Good news is I am no longer getting the empty space characters. Bad news is I am still not saving the earlier writes.

 

I went a little crazy with the rewinds (however, I also removed some places to check for change in effect).

 

I would expect rewind 1 isn't required because w+ automatically goes to the beginning.

 

I also don't think rewind 2 is required based on my reading of the documents.

 

Rewind 3 normally is required, but since I am truncating to 0, I wouldn't think it was.

 

Also looking at the documents, I didn't see any need of using ftruncate after the fwrite.

 

If starting with an empty myjson.json file, if I run the below script once, I get [{"name":"hello3"}].  If I run it twice, I get [{"name":"hello2"},{"name":"hello3"}], and if I run it again and again, I only get what it produced on the second run.
 
Is this expected?  My desired results are [{"name":"hello1"},{"name":"hello2"},{"name":"hello3"}] if I run it once, and [{"name":"hello1"},{"name":"hello2"},{"name":"hello3"},{"name":"hello1"},{"name":"hello2"},{"name":"hello3"}]if I run it twice, and so on.  How can this be accomplished?
 
Thanks
<?php
$file='myjson.json';


addRecord($file,'hello1');
addRecord($file,'hello2');
addRecord($file,'hello3');


echo(file_get_contents($file));


function addRecord($file,$msg) {
    $fp = fopen($file, "w+");
    //$fp = fopen($file, "c+");
    if (flock($fp, LOCK_EX)) {
        rewind($fp);    //rewind 1
        $filesize=filesize($file);
        $json=$filesize?fread($fp, $filesize):'[]';
        rewind($fp);    //rewind 2
        $arr=json_decode($json);
        $newRow=(object)['name'=>$msg];
        $arr[]=$newRow;
        $new_json=json_encode($arr);
        ftruncate($fp, 0);
        rewind($fp);    //rewind 3
        fwrite($fp, $new_json);
        rewind($fp);    //rewind 4
        fflush($fp);
        flock($fp, LOCK_UN);
    }
    else{
        exit('Could not lock the file!!!');
    }
    fclose($fp);
}

 

Link to comment
Share on other sites

The “w+” mode truncates the entire file right from the beginning, so there's nothing to read. You need “r+”.

 

It would help if I learned to read...

 

Open for reading and writing; place the file pointer at the beginning of the file and truncate the file to zero length. If the file does not exist, attempt to create it.

Link to comment
Share on other sites

No change in results.  Do you mind running this script, and see what you get?  Thanks

 

<?php
$file='myjson.json';


addRecord($file,'hello1');
addRecord($file,'hello2');
addRecord($file,'hello3');


echo(file_get_contents($file));


function addRecord($file,$msg) {
    $fp = fopen($file, "r+");
    if (flock($fp, LOCK_EX)) {
        //rewind($fp);    //rewind 1.  Not required since??? 'r+'    Open for reading and writing; place the file pointer at the beginning of the file
        $filesize=filesize($file);
        $json=$filesize?fread($fp, $filesize):'[]';
        rewind($fp);    //rewind 2.  Where is the pointer now?  http://php.net/manual/en/function.fread.php doesn't seem to say.
        $arr=json_decode($json);
        $newRow=(object)['name'=>$msg];
        $arr[]=$newRow;
        $new_json=json_encode($arr);
        ftruncate($fp, 0);
        rewind($fp);    //rewind 3  Per http://php.net/manual/en/function.ftruncate.php, the pointer isn't changed.
        fwrite($fp, $new_json);
        //rewind($fp);    //rewind 4.  Where is the pointer now?  http://php.net/manual/en/function.fwrite.php doesn't seem to say.
        fflush($fp);
        flock($fp, LOCK_UN);
    }
    else{
        exit('Could not lock the file!!!');
    }
    fclose($fp);
}
Link to comment
Share on other sites

  • Solution

filesize

Note: The results of this function are cached. See clearstatcache() for more details.

Your $filesize is only going to be correct the first time the function is called. After that you'll get the cached value back. Either clear the cache or read the file in a way that doesn't depend on obtaining the file size.

Link to comment
Share on other sites

 

        //rewind($fp);    //rewind 1.  Not required since??? 'r+'    Open for reading and writing; place the file pointer at the beginning of the file

 

Correct, the rewind is not necessary, the file pointer begins at the start of the file in that mode.

 

 

        rewind($fp);    //rewind 2.  Where is the pointer now?  http://php.net/manual/en/function.fread.php doesn't seem to say.

 

fread advances the file pointer by the length of the data read, so it will be just past the last byte you read. In your case at the end of the file since you're reading everything.

 

       

 

        //rewind 4.  Where is the pointer now?  http://php.net/manual/en/function.fwrite.php doesn't seem to say.

 

Like fread, fwrite advances the pointer by the amount of data written, so the pointer would be just after the last byte written. In your case that would be the end of the file since you're starting with a blank file.

 

Of the two rewinds you have you only need one of them. ftruncate does not care about the file pointer's position so whether your rewind before or after it doesn't matter from an implementation point of view. I prefer to rewind after it as I feel it makes it more obvious what the code is doing.

 

While not necessary in your code, you might also want to look at fseek, this lets you move the file pointer arbitrarily. That is useful if you're dealing with file formats where you can calculate the offset to data (eg, fixed length records). You can also use it and ftell (or fstat) as an alternative to filesize if you have a pointer to a file but not path.

 

fseek($fp, 0, SEEK_END);
$filesize = ftell($fp);
Edited by kicken
  • Like 1
Link to comment
Share on other sites

This thread is more than a year old.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.