Requesting multiple data requests in a single REST request

[NotionCommotion]

Let's say a REST client needs four pieces of information. It could do it in four requests, and when it receives the data, it knows it is associated with what was requested.

 

1. Sent: {"item":"a", "function":"SUM", "limit":123} Received: {"john":140,"Mary":120}
   
2. Sent: {"item":"a", "function":"MAX", "limit":123} Received: {"john":18,"Mary":12}
   
3. Sent: {"item":"b", "function":"MIN", "limit":123} Received: {"john":4,"Mary":6}
   
4. Sent: {"item":"a", "function":"SUM", "limit":321} Received: {"john":240,"Mary":320}
   

 

Turns out the server's magic formula is $sql="SELECT name, $function($item) AS value FROM my_table WHERE $limit<123 GROUP BY name"; (PS, I won't really do this quite so dangerously).

 

A single database query could return the first three pieces of data, and it seems a shame to do it with three separate queries.  Instead, the server could quickly look at the item and the limit, and dynamically create a query which would provide multiple pieces.

 

But if all data is requested at once, my conundrum is how the server should label the pieces of data so when the client gets it back, it knows which one is associated with which sub-request.  I was thinking of just an array, or maybe hashing the request and using that as the property.

 

Hope this makes at least some sense as I am just starting off learning about REST.

[ginerjm]

Any request must be made using some recognizable piece of data from that record ( I call them records). Thererfore when the data is returned each record s/b identifiable by that piece of data (key?) and thus it can be used accordingly. Does that make sense?

[NotionCommotion]

Any request must be made using some recognizable piece of data from that record ( I call them records). Thererfore when the data is returned each record s/b identifiable by that piece of data (key?) and thus it can be used accordingly. Does that make sense?

 

Thanks ginerjm,  What you are saying makes sense.  I just don't seem to have any such key.  I suppose the client could generate a surrogate for no other purpose other than send it with the request, and then the server could use it to label it for the way back.  Is doing so typically done?  Or I could maybe hash the request and use that as the key?

[Psycho]

If I am understading the question correctly, I would suggest returning the data in a multi-dimensional array (in JSON or whatever format you want to use)

 

E.g.

 

Sent: {"item":"a", "limit":123}
Received: {"john":{"MIN":4,"MAX",18,"SUM",140},"Mary":{"MIN":6,"MAX",12,"SUM",120}}

 

The back-end query would run this

$sql="SELECT name, MIN($item) as minVal, MAX($item) as maxVal, SUM($item) as sumVal
      FROM my_table WHERE $limit<123 GROUP BY name"; 

 

 . . . or maybe I am not understanding the issue.

[ginerjm]

I must not understanding this REST thingy. A request for data simply has to have some identity or else you are just taking data blindly. I've covered the first case already. In the second, if the data is not identifiable, then why would it matter how you handle it once you receive it?

[NotionCommotion]

Thanks Psycho and ginerjm,

 

Originally, I was thinking that the client should provide the request from their point of view, and let the server deal with optimizing the queries.  I was also thinking of returning a given aggregate function only if the client needed it, but it definitely simplifies things to return all regardless of whether requested (for instance, I wasn't requesting MIN of Item "a", but it is not a big deal to send it anyways).  For instance:

Sent: [{"item":"a","fn":"SUM","limit":123},{"item":"a","fn":"MAX","limit":123},{"item":"b","fn":"MIN","limit":123},{"item":"a","fn":"SUM","limit":321}]
Received: [{"john":140,"Mary":120},  {"john":18,"Mary":12},  {"john":4,"Mary":6}, {"john":240,"Mary":320}}

This would require identifying the request with the reply based on each's position in the array, and I was concerned that doing so was a bad idea.

 

Or, I could require that the client structure it, and do something like the following (still need to deal with the limit 321):

Sent: {"item":["a","b"], "limit":123}

Received: {"a":[{"john":{"MIN":4,"MAX",18,"SUM",140}}, {"mary":{"MIN":6,"MAX",12,"SUM",120}}, "b":["john":{"MIN":4,"MAX",12,"SUM",240}},"Mary":{{"MIN":6,"MAX",22,"SUM",420}}}
or
Received: {"a":{"john":{"MIN":4,"MAX":18,"SUM":140},"mary":{"MIN":6,"MAX":12,"SUM":120}},"b":{"john":{"MIN":4,"MAX":12,"SUM":240},"mary":{"MIN":6,"MAX":22,"SUM":420}}}

[Jacques1]

What is the problem you're trying to solve?

 

You've spent a lot of time thinking about implementation details, but why do you want to merge multiple requests into one? Is this supposed to be some kind of optimization? If so, what are the underlying performance issues, and how exactly does your approach solve them?

 

The very first example with four separate requests makes perfect sense to me. The “optimized” requests don't.

Edited June 2, 2016 by Jacques1

[NotionCommotion]

What is the problem you're trying to solve?

 

I must apologize to you, Psycho, and ginergr.  I thought a generic situation would be more relevant to others, but I can't seem to communicate the issues.  Please let me start over.

 

Multiple real-time monitored parameters (i.e. freezer temperature, freezer power, etc) are being stored on 15 second intervals with history which might span over many years.

 

I need to create an API to query the data which is based on the following:

• The parameter being monitored such as freezer #1 temperature.
• An aggregate function on the parameter such as the minimum, maximum, sum, weighted sum, count, average, etc.
• A MaxTime > time > MinTime WHERE statement.  This constraint will actual be something like "from X seconds ago and going back Y seconds".

 

The client will typically ask on ideally a single request "Give me maximum and minimum temperature of freezers #1, #2, and #3 from one year ago to three years ago, minimum temperature of freezer #3, #4 and #5 for those same time spans, and average temperature of freezers 2 and 5  but between today and 4 years ago."

 

The queries might be expensive, and should be optimized.  We don't have total control over the clients request, and would like the server to have some control over its own destiny. I recognize I haven't provided the database schema which makes any definitive answers impossible, but....

 

How should the client request the data and the data be returned so that the server could optimize the queries and the client be able to identify the returned results?

[NotionCommotion]

After giving it more thought, it is the ranges that are giving me problems.

 

For instance, let's say the client wants the following.

maximum freezer_1_temp starting zero years ago (0 s) for a duration of three years (94608000 s)
minimum freezer_1_temp starting zero years ago (0 s) for a duration of three years (94608000 s)
maximum freezer_2_temp starting zero years ago (0 s) for a duration of three years (94608000 s)
maximum freezer_1_temp starting one year ago (31536000 s) for a duration of four years (126144000 s)

One option is that I could send the data described under SENT, and receive the data described under RECEIVED.  While I like this from an object perspective, I really don't like how I am concatenating the ranges to make a property name.  But what else could I do?

 

Another option is I send the same thing (an array with 4 elements), and I return [40, 20, 20, 30].  But this means that the order of the sent and return data identifies each.  I don't know why, but I just feel one shouldn't do so.

 

SENT

Array(    [0] => stdClass Object        (            [param] => freezer_1_temp            [fnc] => max            [range] => Array                (                    [0] => 0                    [1] => 126144000                )        )    [1] => stdClass Object        (            [param] => freezer_1_temp            [fnc] => min            [range] => Array                (                    [0] => 0                    [1] => 126144000                )        )    [2] => stdClass Object        (            [param] => freezer_2_temp            [fnc] => max            [range] => Array                (                    [0] => 0                    [1] => 126144000                )        )    [3] => stdClass Object        (            [param] => freezer_1_temp            [fnc] => max            [range] => Array                (                    [0] => 31536000                    [1] => 126144000                )        ))

RECEIVED

stdClass Object(    [freezer_1_temp] => stdClass Object        (            [0_126144000] => stdClass Object                (                    [max] => 40                    [min] => 20                )            [31536000_126144000] => stdClass Object                (                    [max] => 30                )        )    [freezer_2_temp] => stdClass Object        (            [0_126144000] => stdClass Object                (                    [max] => 20                )        ))