Help with sockets and ReactPHP

[NotionCommotion]

I am trying to communicate between two machines using sockets and the http://reactphp.org/ framework.  For now, the client is just sending data and I know I can use cURL, but will later need the server to initiate communication, so please just assume I want to use sockets.  The data transferred is below is just a string of dots, but will eventually be JSON.

 

I ran the below client2.php script on a Raspberry Pi (192.168.1.210) and the below server2.php script on a Centos box (192.168.1.200), and got the results shown under TEST 1.  I then swapped the two machines (and swapped the value of $ip in each), and got the results under TEST 2.

 

Questions...

1. Why am I loosing data?  Is my loop rate two fast (1 second)?  I've changed it to 3 seconds, and it happens less but still happens. I suppose I can add some sort of checksum to the data, but surely doing so isn't required with TCP?
2. Why am I receiving more packets than I am sending?  They seem to add up to the correct amounts that are being sent (so maybe this addresses my Question 1).  How does the server "know" which packets need to be combined?  It is JSON, and half a JSON string doesn't mean much.
3. Why doe the Pi not return true for $connect->write($package) when the string is 2048 or greater?  What does this really mean as you can see the other machine is still getting the data.
4. How can the client "know" that the server received a package of data?
5. Don't know if I wish to ask this, but what else am I doing wrong with my implementation?

Thanks!

 

 

client2.php

<?php
// Client
require 'vendor/autoload.php';

use React\EventLoop\Factory;
use React\Socket\Connection;

ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
set_time_limit(0);

$ip='192.168.1.200';

$loop = Factory::create();
$server = @stream_socket_client("tcp://$ip:1337", $errno, $errstr, STREAM_CLIENT_ASYNC_CONNECT);
echo 'Connection successful!' . PHP_EOL;

$socket = new Connection($server, $loop);
//$socket->bufferSize = 65536;
$connect=$socket;

$socket->on('data', function($data) {
    echo "on data: $data" . PHP_EOL;
});

$socket->on('close', function() {
    echo "on close" . PHP_EOL;
});

$loop->addPeriodicTimer(1, function() use ($connect) {
    static $counter=2040;
    $package=str_repeat('.',1*$counter). PHP_EOL;
    $counter++;
    $size=strlen($package);
    if($connect->write($package)) {
        echo "Message Sent ($size)". PHP_EOL;
    }
    else {
        echo "Message Not Sent ($size)". PHP_EOL;
    }
});

echo 'Start Client ...' . PHP_EOL;
$loop->run();

server2.php

<?php
// Server
require 'vendor/autoload.php';

use React\EventLoop\Factory;
use React\Socket\Server as SocketServer;
use React\Http\Server as HttpServer;

ini_set('display_errors', 1);
ini_set('display_startup_errors', 1);
error_reporting(E_ALL);
set_time_limit(0);

$ip='192.168.1.200';

$loop = Factory::create();

$socket = new SocketServer($loop);
$socket->on('connection', function($conn) {

    $conn->on('data', function($data) {
        $size=strlen($data);
        echo "on data: ($size)" . PHP_EOL;
    });

    $conn->on('close', function($conn) {
        echo "on close" . PHP_EOL;
    });

    $conn->on('error', function($conn) {
        echo "on error" . PHP_EOL;
    });

});
$socket->listen(1337,$ip);
$loop->run();

******************************* TEST 1 (Pi as client and Centos as server) *******************************************

michael@raspberrypi:/var/www $ php client2.php
Connection successful!
Start Client ...
Message Sent (2041)
Message Sent (2042)
Message Sent (2043)
Message Sent (2044)
Message Sent (2045)
Message Sent (2046)
Message Sent (2047)
Message Not Sent (2048)
Message Not Sent (2049)
Message Not Sent (2050)
^C
michael@raspberrypi:/var/www $

[Michael@devserver react]$ php server2.php
on data: (2041)
on data: (2042)
on data: (2043)
on data: (1448)
on data: (596)
on data: (1448)
on data: (597)
on data: (1448)
on data: (598)
on data: (2047)
on data: (2048)
on data: (1448)
on data: (601)
on data: (2050)
on close
^C
[Michael@devserver react]$

******************************* TEST 2 (Centos as client and Pi as server) *******************************************

[Michael@devserver react]$ php client2.php
Connection successful!
Start Client ...
Message Sent (2041)
Message Sent (2042)
Message Sent (2043)
Message Sent (2044)
Message Sent (2045)
Message Sent (2046)
Message Sent (2047)
Message Sent (2048)
Message Sent (2049)
Message Sent (2050)
Message Sent (2051)
Message Sent (2052)
Message Sent (2053)
Message Sent (2054)
Message Sent (2055)
Message Sent (2056)
Message Sent (2057)
^C
[Michael@devserver react]$

michael@raspberrypi:/var/www $ php server2.php
on data: (2041)
on data: (2042)
on data: (2043)
on data: (2044)
on data: (2045)
on data: (2046)
on data: (2047)
on data: (2048)
on data: (2049)
on data: (1448)
on data: (602)
on data: (2051)
on data: (2052)
on data: (2053)
on data: (2054)
on data: (2055)
on data: (2056)
on data: (2057)
on close
^C
michael@raspberrypi:/var/www $

[NotionCommotion]

A couple other observations.  While $connect->write() returns false on the Pi returns for packages 2,048 and greater, Centos does at 65,536 and greater.  Sounds like a configuration issue.  Still, the question remains what the function returning true or false really means, and how the client can know whether the server received the data.

 

Also, big packages are always broken up in smaller packages of around 1 to 2K. So, with my small data transfers, I just witnessed this every now and then.  Somehow, the server script needs to determine which packages go together and to combine them

[Jacques1]

TCP is dealing with streams. That is, you send and receive a sequence of raw bytes. There is no concept of “messages” or anything like that, so turning the byte stream back into JSON documents is your job.

 

How the strings you send are split into TCP segments is up to the operating system (see the TCP protocol). There's no guarantee whatsoever that a single PHP string will be sent in one go. You may have to repeat the sending procedure multiple times until all data has been acknowledged.

 

As to the return value of the write() method, see the documentation.

 

A fair warning: Sockets are low-level constructs. This is not like HTTP where you (mostly) get the complete content and can just access the data.

[NotionCommotion]

Thanks Jacques1,

 

After driving a bit and thinking it over, I better realized that sockets deal with streams of TCP segments.  Are each of these segments sent individually, and might go through different Internet routers to eventually get to their destination, and is that why they might be received in a different order?

 

But receiving data in arbitrary order seems pretty useless.  Somehow it must be reassembled after received in some intelligent order.  I would think the reactphp class would handle this, and am wondering if something like $conn->on('end', function($data) {…}); instead of 'data' can be used?

 

Also, after looking over your referenced documentation, write($data) will return false only if the supplied $data which might need to be buffered exceeds the internal buffer size, and it has nothing to do whether the receiving machine actually received the data.  Do you think I will need to implement my own homemade acknowledgement protocol where after the receiving receives the data issues another stream back saying it has been acknowledged?  This sounds a bit challenging as I wish to remain asynchronous and wouldn’t want to use some blocking function, and again would have expected the class to somehow implement this. 

 

Thanks again

[Jacques1]

After driving a bit and thinking it over, I better realized that sockets deal with streams of TCP segments.  Are each of these segments sent individually, and might go through different Internet routers to eventually get to their destination, and is that why they might be received in a different order?

 

The receiver will not get data in the wrong order. TCP is explicitly designed as a reliable protocol, which means you will get the complete data in the right order. If a segments gets lost, it's resent, if segments are received in the wrong order (for whatever reason), they're reordered according to their sequence numbers.

 

That's why I said that you're conceptually dealing with a byte stream. The segments are just implementation details which are handled by the operating system.

 

 

 

But receiving data in arbitrary order seems pretty useless.  Somehow it must be reassembled after received in some intelligent order.  I would think the reactphp class would handle this, and am wondering if something like $conn->on('end', function($data) {…}); instead of 'data' can be used?

 

No. TCP streams by definition don't end unless you close the entire connection.

 

 

 

Also, after looking over your referenced documentation, write($data) will return false only if the supplied $data which might need to be buffered exceeds the internal buffer size, and it has nothing to do whether the receiving machine actually received the data.  Do you think I will need to implement my own homemade acknowledgement protocol where after the receiving receives the data issues another stream back saying it has been acknowledged?  This sounds a bit challenging as I wish to remain asynchronous and wouldn’t want to use some blocking function, and again would have expected the class to somehow implement this. 

 

The class does implement this. That's actually the whole point of the buffer: It holds the data until it has been sent and acknowledged by the receiver.

 

What you do is pass a JSON document (or chunks of thereof) to the write() method. If the buffer is full, you have to wait for the drain event to send the next document (or chunk) etc.

[NotionCommotion]

Jacques1,  I really wish you would just write my damn code for me and stop trying to teach me.  Well, not really, but sometimes...  I do really appreciate all the help I have received on this forum, and have learned a great deal.

 

How the strings you send are split into TCP segments is up to the operating system (see the TCP protocol). There's no guarantee whatsoever that a single PHP string will be sent in one go. You may have to repeat the sending procedure multiple times until all data has been acknowledged.

"You may have to repeat the sending procedure multiple times..."  You being me/PHP or TCP and/or the operating system?

 

 

The class does implement this. That's actually the whole point of the buffer: It holds the data until it has been sent and acknowledged by the receiver.

 

write($data): Write some data into the stream. If the stream cannot handle it, it should buffer the data or emit and error event. If the internal buffer is full after adding $data, write should return false, indicating that the caller should stop sending data until the buffer drains.

When you say "and acknowledged by the receiver", which method?  I would have expected it to be write(), but per https://github.com/reactphp/stream#methods-1, write only returns true if the buffer is not exceeded, and else false.  How do I know that it has been received?

 

 

Okay, I understand what you are saying regarding sending chunks and waiting for the drain event.  But just to make sure, this is just to stay within the buffer, right?  I am still not sure how the receiver deals with knowing when it has received all the data and can interpret it as JSON.  I suppose I can concatenate the incoming text stream and check if it is valid JSON, however, I hope this is not the way to do so.

 

 

PS.  I was a couple months behind on the reactphp code on the Pi, and after updating composer, now I witness the same 65,536 buffer as the Centos box

[kicken]

After driving a bit and thinking it over, I better realized that sockets deal with streams of TCP segments.  Are each of these segments sent individually, and might go through different Internet routers to eventually get to their destination, and is that why they might be received in a different order?

 

But receiving data in arbitrary order seems pretty useless.  Somehow it must be reassembled after received in some intelligent order.  I would think the reactphp class would handle this, and am wondering if something like

 

 

Yes, each packet can take a completely different route across the internet. Due to lag and what not, yes they can be received out of order by the target system. All this however is handled by the TCP protocol and not something you need to worry about yourself. By the time the data reaches your application it is guaranteed to be intact and in-order.

 

When TCP splits the data into packets and sends them each packet sent is tagged with a sequence number. The receiving end tracks which sequence numbers it has received and re-arranges the received data into the correct order if necessary. In addition it will send back and ACKnowledgement packet to the sender telling it that the data has been successfully received. If the sender doesn't get an ACK for a given packet it will keep trying to re-transmit that packet until it's received successfully.

 

If you're interested in more details about how it all works I'd suggest you read up on the TCP protocol and maybe install something like Wireshark and monitor some actual traffic.

 

"You may have to repeat the sending procedure multiple times..."  You being me/PHP or TCP and/or the operating system?

Both, to some extent. The OS has a buffer that will hold data until it can be successfully sent. If that buffer is full then attempting to send more data will fail. For data that the OS has already accepted and placed into this buffer it will automatically re-transmit as necessary until it receives an ACK.

 

When the buffer is full and a write would fail one of two things could happen. If your socket is in blocking then the OS will sleep for a bit and try again later. This results in your program being paused on the write() call until the data can be sent (or some other error occurs). If your socket is non-blocking then the OS will return an error code indicating the buffer is full and you need to try again. In this instance it's up to your program to keep track of the data and attempt to send it again later.

 

React uses non-blocking sockets and handles this buffering of written data for you. If write() returns false it means that react's buffer is full and you should not write any more data until after a drain event is received, however looking at the code it appears as though it currently doesn't actually prevent you from adding more data.

 

I am still not sure how the receiver deals with knowing when it has received all the data and can interpret it as JSON.

That's something you have to build into your protocol. For example HTTP knows what is headers vs content by looking for the first "\r\n\r\n" sequence.

 

Some other methods involve pre-pending the length of the content, for example you could write data such as "$length\n$json" where $length is strlen($json). On the receiving end you'd read the first line to get the length then read $length bytes more from the stream.

Edited December 29, 2016 by kicken

[Jacques1]

I am still not sure how the receiver deals with knowing when it has received all the data and can interpret it as JSON.  I suppose I can concatenate the incoming text stream and check if it is valid JSON, however, I hope this is not the way to do so.

 

That is pretty much the way. There is no inherent the-JSON-document-has-been-received state in TCP. You have to concatenate the JSON fragments you get and either run them through a parser which supports streaming or separate the documents with a special character like a newline or use a length prefix in front of each document. The “finished” state must be implemented by you at application-level.

 

If you want the sender to know when a document has been received: It doesn't. The write() method passes the data to a React buffer, React passes the content to a kernel buffer, and then the operating system takes over to actually generate and send the TCP segments (which may require many attempts and some time). Everything else is out of your control. You could make the receiver send you some kind of acknowledgement for each JSON document, but that won't be very useful.

Edited December 29, 2016 by Jacques1

[NotionCommotion]

Thanks Kicken,  Helped fill a few, few, and few gaps of mine.

Yea, reading the TCP protocol is on my bucket list.  I really need to do so.  A while back, I played with Wireshark, but need to do more.

Regarding this ACKnowledgement packet, does react and/or PHP get this information?  Wouldn't this provide confirmation that something was received?

Your comments about needing to resend data, buffering, drain, etc makes sense.  You are correct that I didn't not implement drain/etc as at the time I did not understand such a concept existed, but will implement them.
 
As far building a protocol and implementing things like "\r\n\r\n" and "$length\n$json", really?  I for one seem to like home building my own stuff, but I don't think I should do so.  Is what I am attempting to do so really so unique?  Many clients need to initiate communication to a server, and the server must be able to receive data from the clients as well as send data to a specific client.  I would expect at least a spec and even several implementation classes exist to do so.

 

Thanks again

 

PS.  2016 kind of sucked, and wishing all a good 2017!

[NotionCommotion]

That is pretty much the way. There is no inherent the-JSON-document-has-been-received state in TCP. You have to concatenate the JSON fragments you get and either run them through a parser which supports streaming or separate the documents with a special character like a newline or use a length prefix in front of each document. The “finished” state must be implemented by you at application-level.

 

If you want the sender to know when a document has been received: It doesn't. The write() method passes the data to a React buffer, React passes the content to a kernel buffer, and then the operating system takes over to actually generate and send the TCP segments (which may require many attempts and some time). Everything else is out of your control. You could make the receiver send you some kind of acknowledgement for each JSON document, but that won't be very useful.

 

Thanks Jacques1, but not what I wanted to hear.  The "confirm valid JSON" approach seems a little prone for errors.  First, it needs to smell at the start like JSON.  Okay, easy enough.  But then the concatenated string needs to be JSON, or the concatenated x N  strings need to be JSON.  I see things getting exponential really quick!  In regards to the newline/length prefix, same concerns that I responded to Kicken.

 

As far acknowledgement, that client/sender holds data, sends it to the server/receiver, and if received, doesn't attempt to send it any more and even deletes the data.  As such, I think it is important to know that it is received.  Maybe I shouldn't use low level sockets for this and instead use curl?  My understanding is curl is just sockets but with more overhead protocol.

[Jacques1]

Regarding this ACKnowledgement packet, does react and/or PHP get this information?

 

No. TCP acknowledgements are processed deep inside the operating system. You cannot even see them at the system-call level below PHP.

 

 

 

As far building a protocol and implementing things like "\r\n\r\n" and "$length\n$json", really?  I for one seem to like home building my own stuff, but I don't think I should do so.  Is what I am attempting to do so really so unique?  Many clients need to initiate communication to a server, and the server must be able to receive data from the clients as well as send data to a specific client.  I would expect at least a spec and even several implementation classes exist to do so.

 

Newline-delimited JSON is a de-facto standard. I've also seen (but never tried) several stream-based JSON parser for PHP which can extract documents from byte streams.

 

Like I said, sockets are low-level concepts. If you expect this to be super-easy, you've picked the wrong approach.

 

 

 

As far acknowledgement, that client/sender holds data, sends it to the server/receiver, and if received, doesn't attempt to send it any more and even deletes the data.  As such, I think it is important to know that it is received.

 

Why would the sender resend the data? I think you're still missing the fact that TCP is a realiable protocol which takes care of all those details. You never have to worry about duplicate or lost data. You just push the bytes of your JSON documents into the sending buffer and read them from the receiving buffer.

 

 

 

Maybe I shouldn't use low level sockets for this and instead use curl?  My understanding is curl is just sockets but with more overhead protocol.

 

When you're talking about cURL, you probably mean HTTP. Yes, HTTP is a protocol on top of TCP, yes, it has some overhead. But it hides all the ugly details, and you should be very familiar with it.

 

With HTTP, you can just send a JSON document. Not a stream of bytes.

[NotionCommotion]

No. TCP acknowledgements are processed deep inside the operating system. You cannot even see them at the system-call level below PHP.

Bummer

 

Newline-delimited JSON is a de-facto standard.

Perfect!

 

Why would the sender resend the data? I think you're still missing the fact that TCP is a realiable protocol which takes care of all those details. You never have to worry about duplicate or lost data. You just push the bytes of your JSON documents into the sending buffer and read them from the receiving buffer.

 

But what if sender sends the data, but receiver gets blown up?  Sorry, but I had several typos regarding this part.  Sender gets local data and sends it remotely.  If the data is not received remotely, it stores it locally so it can be sent remotely later.  If the stored data is finally delivered, it deletes the local data.  I still need to know that it was received. 

 

When you're talking about cURL, you probably mean HTTP. Yes, HTTP is a protocol on top of TCP, yes, it has some overhead. But it hides all the ugly details, and you should be very familiar with it.

 

With HTTP, you can just send a JSON document. Not a stream of bytes.

 

Yes, HTTP.  My "big" data can be sent via HTTP as client is behind a firewall and will know the IP which it will be sent to.  My only need for sockets is for the server to send the client little data.  You think it makes sense to use HTTP/cURL for all client to server communication?  Sorry, disregard this as I am sure I haven't provided enough information to make an informed recommendation, but I (now) think this is the way to go.  This would also provide headers for confirmation of reception.

[Jacques1]

But what if sender sends the data, but receiver gets blown up?  Sorry, but I had several typos regarding this part.  Sender gets local data and sends it remotely.  If the data is not received remotely, it stores it locally so it can be sent remotely later.  If the stored data is finally delivered, it deletes the local data.  I still need to know that it was received.

 

No, you need to know if it was successfully processed by the target application, which is much more than a simple TCP acknowledgement. If you want an application-level success message, you have to implement it yourself (as stated earlier). That is, the JSON receiver must send its own data to tell the sender that the document can be deleted.

 

 

 

Yes, HTTP.  My "big" data can be sent via HTTP as client is behind a firewall and will know the IP which it will be sent to.  My only need for sockets is for the server to send the client little data.  You think it makes sense to use HTTP/cURL for all client to server communication?  Sorry, disregard this as I am sure I haven't provided enough information to make an informed recommendation, but I (now) think this is the way to go.  This would also provide headers for confirmation of reception.

 

HTTP is a good all-around tool, and JSON over HTTP is very common. Yes, I think it's the right choice when there are no special requirements. If traffic ever becomes an issue, you can still use compression (which is very effective for text-based formats).

[kicken]

Many clients need to initiate communication to a server, and the server must be able to receive data from the clients as well as send data to a specific client.  I would expect at least a spec and even several implementation classes exist to do so.

Many specs and implementations do exist, for example HTTP and WebSocket. If you don't want to use any of the existing protocols though then you need to develop your own. Developing your own protocol involves solving problems like how to know when a complete message has arrived, how to distinguish between different messages, how to detect failures, etc.

 

If you'd rather not deal with designing a protocol of your own, use one of the existing ones. HTTP is popular because it's very flexible and is widely supported. The main disadvantage to HTTP is that generally uni-directional and stateless. WebSocket is fairly new still but I suspect it will gain popularity as a generic bi-directional protocol with good library support.

[NotionCommotion]

Kicken, If bi-directional communication is required, I too was thinking that websockets is likely the way to go as I really, really don't want to develop my own and with the wide attention placed on it, expect it is rather robust.  My understanding (correct me if I am wrong) is websockets was developed likely specifically for browser clients to remote server bi-directional communication, right?  That being said, it probably still would be a good choice for server-to-server bi-directional communication, agree?

 

Jacques1,  I would have hoped that reactphp would have implemented some of this application authentication.  I do agree that HTTP seems like the best choice "when" it could be used, and otherwise fall back to some more low-level solution.

 

Thanks for both your help.

[Jacques1]

Jacques1,  I would have hoped that reactphp would have implemented some of this application authentication.

 

The React classes are explicitly designed as (relatively simple) wrappers for the PHP stream functions to implement client-server communication over sockets. Anything beyond this is out-of-scope.

[NotionCommotion]

The React classes are explicitly designed as (relatively simple) wrappers for the PHP stream functions to implement client-server communication over sockets. Anything beyond this is out-of-scope.

 

Do you know of any good classes to handle application authentication and Newline-delimited JSON (or something similar)?