Converting collection of objects to JSON

[NotionCommotion]

I can obtain either one or a collection of objects by executing the following.

$product = $entityManager->find('Product', $id);
$products = $entityManager->getRepository('Product')->findAll();

All products have many common properties, however, based on the type of product there are some differences.  To implement this, I am using class table inheritance (side question - is there any difference between CTI and supertype/subtype modeling?).

I will need to convert either a single product to JSON as well as a collection of products to JSON.  When converting a collection, only a subset of the product's properties is desired, however, again most of these properties are common to each product type but there are some differences.

One solution is to utilize the JsonSerializable interface.  A shortcoming, however, is how to deal with both the detail view with all the properties and the collection view with only a subset of properties.  To overcome this, I can potentially use jsonSerialize() to provide the detail view, and add another function index() to provide the subset results for the collection.  Since this index() function will not recursively incorporate sub-objects, I will need to manually loop over the collection, however, it should not be too big of issue.

Or should the entity not be responsible to convert to JSON?  I am kind of mixed on this issue, but do agree using a separate class has merit.  So, if a separate class is used, how does it deal with each subclass of Product which might have unique properties and getters?  I suppose I can have the entities implement an interface which requires them to return their applicable properties, but doing so isn't very flexible and isn't this (almost) the same as having the entity responsible to provide the JSON?  I "could" make this other class perform some switch operation based on the class name, but doing so goes against all good OOP principles.

What is the "right" way to do this?

Thanks

[kicken]

If you need different views for the same object then I'd suggest creating a separate service that handles that.  Pass it the object(s) you want to serialize and the type of view you want and let it handle the serialization rather than the entity with JsonSerializable.

For example, maybe write a class that takes an object and a JSON Schema and will serialize the object according to that schema.  Then you can just create a different schema for each data view you need.  I looked into doing something like this once, but ended up going a different direction that made it not necessary.

 

 

[NotionCommotion]

Thanks kicken,

JSON Schema looks interesting, but I am not sure it is an answer.

Let's say we have a collection (or array) containing 6 objects.

Array
(
    [0] => ProjectC Object
        (
            [c:protected] => 5
            [g:protected] => 5
            [h:protected] => 2
            [a:protected] => 1
            [b:protected] => 3
            [d:protected] => 5
        )

    [1] => ProjectA Object
        (
            [b:protected] => 4
            [d:protected] => 2
            [e:protected] => 7
            [f:protected] => 8
            [a:protected] => 1
            [c:protected] => 2
        )

    [2] => ProjectB Object
        (
            [bDiffName:protected] => 1
            [c:protected] => 4
            [d:protected] => 2
            [g:protected] => 3
            [a:protected] => 1
            [b:protected] => 3
        )

    [3] => ProjectB Object
        (
            [bDiffName:protected] => 1
            [c:protected] => 4
            [d:protected] => 2
            [g:protected] => 3
            [a:protected] => 1
            [b:protected] => 3
        )

    [4] => ProjectC Object
        (
            [c:protected] => 5
            [g:protected] => 5
            [h:protected] => 2
            [a:protected] => 1
            [b:protected] => 3
            [d:protected] => 5
        )

    [5] => ProjectA Object
        (
            [b:protected] => 4
            [d:protected] => 2
            [e:protected] => 7
            [f:protected] => 8
            [a:protected] => 1
            [c:protected] => 2
        )

)

 $a should not be included in any of the JSON output.  ProjectB uses $bDiffName instead of $b. Some objects provide have additional properties which should be included.  As such, the desired output should be as shown.

[
	{"b":3,"c":5,"d":5,"g":5,"h":5},
	{"b":4,"c":2,"d":2,"e":7,"f":8},
	{"b":1,"c":4,"d":2,"g":3},
	{"b":1,"c":4,"d":2,"g":3},
	{"b":3,"c":5,"d":5,"g":5,"h":5},
	{"b":4,"c":2,"d":2,"e":7,"f":8}
]

Sure, I can write a class to present the results as needed.  But if so, each object needs its own, no?  And if i write a separate class for each, without making the entity responsible to identify the applicable class or without making some map between entity class name and serializing service, how can the script know which class to use?

Reference code only...

<?php

abstract class Project implements JsonSerializable
{
    protected $a=1, $b=3, $c=2, $d=5;
    public function getB(){return $this->b;}
    public function getC(){return $this->c;}
    public function getD(){return $this->d;}
}

class ProjectA extends Project
{
    protected $b=4, $d=2, $e=7, $f=8;
    public function getE(){return $this->e;}
    public function getF(){return $this->f;}
    public function jsonSerialize() {
        return ['b'=>$this->getB(),'c'=>$this->getC(),'d'=>$this->getD(),'e'=>$this->getE(),'f'=>$this->getF()];
    }
}

class ProjectB extends Project
{
    protected $bDiffName=1, $c=4, $d=2, $g=3;
    public function getBDiffName(){return $this->bDiffName;}
    public function getG(){return $this->g;}
    public function jsonSerialize() {
        return ['b'=>$this->getBDiffName(),'c'=>$this->getC(),'d'=>$this->getD(),'g'=>$this->getG()];
    }
}
class ProjectC extends Project
{
    protected $c=5, $g=5, $h=2;
    public function getG(){return $this->g;}
    public function getH(){return $this->g;}
    public function jsonSerialize() {
        return ['b'=>$this->getB(),'c'=>$this->getC(),'d'=>$this->getD(),'g'=>$this->getG(),'h'=>$this->getH()];
    }
}

$arr=[
    new ProjectC,
    new ProjectA,
    new ProjectB,
    new ProjectB,
    new ProjectC,
    new ProjectA,
];

echo('<pre>'.print_r($arr,1).'</pre>');
echo('<pre>'.json_encode($arr).'</pre>');

 

[kicken]

I don't understand why you have your $bDiffName and getBDiffName() instead of just overriding getB().  If both properties serialize as 'b' I don't see the point in separating them.

If the JSON Schema doesn't work for you that's fine.  What you could do is just define your own JsonSerilzable like interface that your entities implement.  You could either have separate methods to create your different representations or one method that you pass some parameters too in order to determine the proper serialization.

 

[NotionCommotion]

I suppose I too don't understand why I am not just overriding getB().  My need is just to allow it to be flexible enough should two classes be slightly different.

I thought the intent was for the JSON Schema to validate JSON documents.  I will read in more detail.  I actually wrote my own classes to validate JSON and wish I learned about this sooner.

If I understand you correctly, I will need to have the entities responsible for obtaining the data included in the JSON but just should use a separate method instead of the JsonSerialize().  I got schooled a while back by ignace that a separate class should always be responsible to do so and was looking for ways to do so.

[kicken]

1 hour ago, NotionCommotion said:

I thought the intent was for the JSON Schema to validate JSON documents.

It is, but the idea is that you could use that to guide your serialization.

You'd create a schema that describes the end result of the JSON output you need.  That schema would then end up list the objects and properties that make up that json output.  You could then write a class that reads that data from the schema and finds those properties on the objects you pass it to generate the JSON output.

When I was looking into a while back it that was essentially what I was thinking of doing.  I had an API that accepted and responded with JSON.  I was going to define a schema for each endpoint to validate that POST'ed json data was valid.  The idea was to then also serialize responses by using the schema.  I never got that far with it though as things changed.

 

1 hour ago, NotionCommotion said:

If I understand you correctly, I will need to have the entities responsible for obtaining the data included in the JSON but just should use a separate method instead of the JsonSerialize().

You don't have to do it within your entity.   I wouldn't say there's anything wrong with it in this case though.  The part of the code that is part of your entity would just be returning the data that should be serialized.  The actual serialization process would still be in a separate service class.

Like most things it's a trade off between what gets the job done what what might be an ideal solution.  The ideal solution might involve a number of classes and mapping files to configure what data to pull from what objects.  That'd take time to design an implement though.  A service that just asks the entity directly for the data would be simpler and easier to implement, but maybe less flexible and muddies up the entities a bit.

I find a lot of the times it's better to start with the less ideal solution that works so you can get a better understanding of what you actually need to do (vs what you think you'll need to do) then you can re-factor that into a better solution later on.

[gizmola]

I am going to guess that you are doing this in order to provide a REST api?

Having done this a number of times in the not too distant past, I wired this sort of logic into the controller class.  If you subscribe to MVC, which is easy enough to do this via adoption of the symfony framework, then it's fairly easy to create a base controller class that inherits from the Symfony controller class, and into this you can build in things that make your REST api easy to deal with.   

I mention symfony because you were already using Doctrine if I'm not mistaken, and Symfony already defaults to use of Doctrine.

Once you go down this path, then you have the JMS Serializer bundle which handles serialization of complicated object trees into either of json or xml.  

Even if you don't use symfony, you can still use the Serializer library by itself. 

[NotionCommotion]

@kicken  As always, good advice.  Thank you.

@gizmola  Yes, a REST api.  No, I am not and haven't ever tried Symfony (well, I guess I use Twig).  Using a formal ORM for the first time.  While I like a lot of Doctrine's capabilities, I really don't know yet if I am glad I am trying to adapt it.  Actually, I know I am not glad but hope/expect that will change.  I probably should look into Symfony, but this Doctrine thing has taken me for a loop.  Good news is doing so has allowed me to actually understand some previously foreign concepts that smart programmers on this forum believe are important.  Thanks

[gizmola]

ORM's are ORM's -- they can be great if you drink the coolaid and accept their limitations and quirks.  I do have to say that I far prefer Doctrine2's Data Mapper pattern to Active Record which is used by most of the other framework ORM's and with Ruby on Rails for example.

This is a decent article that touches upon how the Data Mapper pattern is different (and better in my opinion) than Active Record.  In general Doctrine2 is more sophisticated and attempts to do much more than the other PHP ORM's.  Some of the things you have been attempting to do that involve inheritance aren't even remotely possible in other ORM's without you essentially hacking them.

However, relational databases don't provide or support inheritance of any sort, so you are inevitably trying to force a round peg into a square hole.  Probably a decade ago "Object databases" were the hot buzzword, and yet it says a lot that none of the many companies and products emerged with anything that gained a critical mass outside of niche applications tightly bound to specific oop languages.  Meanwhile RDMBS's continue to dominate persistent data storage.

 

[NotionCommotion]

Thanks gizmola,  I certainly hope I am drinking the doctrine coolaid for a good reason!

RDBs "kind of" support inheritance using supertype/subtype, don't you think?

I was looking closer at your referenced serializer link.  First of all, other than dealing with recursion, what is the point of using it instead of plain old json_encode()?  Secondly, the documentation states that the most common use is serializing an object, but when I try the following, somehow the script just ends and returns nothing (I haven't totally tracked it down yet).  Don't know if it matters but I am using xml to define my doctrine entities.  Maybe I need to define what it returns?

$serializer = \JMS\Serializer\SerializerBuilder::create()->build();
$serializer->serialize($someEntity, 'json')

 

[gizmola]

Well, no I don't think that RDBMS's support subtypes, although you can model something involving multiple tables that sorta supports it.  I have certainly done this in the past, however, my reason had more to do with wanting to write generic code than it did traditional OOP.    For example, I had a model with an entity table that was essentially the base of a class.  Entities could be these types: member, channel, vendor, stream.  

So looking at that, it's not very oop, but relationally it had some major advantages.    For example, I required a lot of sophisticated grouping.  I had groups that for example, defined for a Member their channels.  For a particular channel there could be any number of streams.  And for each channel there were groups of admins, moderators, etc.  

Because I had built this around a base entity table, as soon as I had put in place the grouping tables, and written code to support these things, I already had essentially created what I needed to relate these different types of things together whether that be as members to channels, or channels to streams, or vendors to channels.  

Then there were assets that could be attached to these things, and again, with a generic set of tables that allowed relation of an asset to an entity, it didn't matter if it was a photo in a stream, or a cover image for a channel, or a video for a product sold by a vendor.

Once i created comments and tags, I could comment on or tag anything with tremendous reuse of code and DRY.

As for serializer,  there are several things it does that are sophisticated, and fit Doctrine really well.  First it goes through a complicated object tree and figure out how to turn that into a sensible data structure.  There is also support for annotating a model to include/exclude particular attributes in your model.  For example, you might not want a rest api to disclose internal key values, and with Serializer, you can annotate your model and serializer will automagically support that for you.

Your error was simple, and right off the documentation page really...

$output = $serializer->serialize($someEntity, 'json');
echo $output;

Of course, if you're just testing this, you want to set the content-type of the page:

header('Content-Type: application/json');

 

 

[NotionCommotion]

Thanks gizmola,

My model example very closely describes mine, and I think it works great for all the reasons you state.  My only issue is determine how to act on an entity when the entity type is not known.  For instance, it would be easy to serialize an array of vendorStreams or an array of channelStream, but not necessarily so for an array of streams which contain both vendorStreams and channelStream.  While I was thinking of making the entities implement an interface which requires a method for each desired output, but it really doesn't belong there for multiple reasons which I finally understand.  Maybe would be okay to making the entities implement an interface for one method which requires an argument which determines the serialization output.  Or better yet, maybe your serializer suggestion is the answer.

7 hours ago, gizmola said:

As for serializer,  there are several things it does that are sophisticated, and fit Doctrine really well.  First it goes through a complicated object tree and figure out how to turn that into a sensible data structure.  There is also support for annotating a model to include/exclude particular attributes in your model.  For example, you might not want a rest api to disclose internal key values, and with Serializer, you can annotate your model and serializer will automagically support that for you.

Your error was simple, and right off the documentation page really...

$output = $serializer->serialize($someEntity, 'json');
echo $output;

 

No, that was not my error and I of course tried it.  I am pretty sure it is because my entities do not have annotation in the PHP class but in a separate XML file.  Looking for documentation how to implement.  If so, then the serialization specification is tied to the entity making the ability to serialize an array of mixed entities simple, yet does so in a way to make it more flexible that just adding individual serialization type methods to each entity.  Well, at least I hope so!

[NotionCommotion]

48 minutes ago, NotionCommotion said:

No, that was not my error and I of course tried it.  I am pretty sure it is because my entities do not have annotation in the PHP class but in a separate XML file.  Looking for documentation how to implement.  If so, then the serialization specification is tied to the entity making the ability to serialize an array of mixed entities simple, yet does so in a way to make it more flexible that just adding individual serialization type methods to each entity.  Well, at least I hope so!

No, it is not because I am using XML metadata for my entities, but because my objects are too large and/or have too much recursion and PHP times out after 20+ seconds.  Looks like I need to figure out how to instruct the serializer only to include the specific properties which I want.

[NotionCommotion]

Not complete, but I definitely think I am on the right path.  Appreciate the help.  I am sure I am not doing the discriminator part correct, and if anyone has experience, would appreciate some additional direction.

<?xml version="1.0" encoding="UTF-8" ?>
<serializer>
    <class name="NotionCommotion\App\Domain\Entity\Chart\Chart" discriminator-field-name="dtype">
        <xml-namespace prefix="atom" uri="http://www.w3.org/2005/Atom"/>
        <discriminator-class value="pie_chart">NotionCommotion\App\Domain\Entity\Chart\Pie\PieChart</discriminator-class>
        <discriminator-groups>
            <group>index</group>
            <group>detail</group>
        </discriminator-groups>
        <property name="account" exclude="true"/>
        <property name="id" exclude="true"/>
        <property name="name" groups="index, detail"/>
        <property name="id_public" serialized-name="id" groups="index, detail"/>
        <property name="series" groups="detail"/>
        <property name="dtype" exclude="true"/>
    </class>
</serializer>

<?xml version="1.0" encoding="UTF-8" ?>
<serializer>
    <class name="NotionCommotion\App\Domain\Entity\Chart\Pie\PieChart" discriminator-field-name="dtype">
        <xml-namespace prefix="atom" uri="http://www.w3.org/2005/Atom"/>
        <discriminator-class value="pie_chart">NotionCommotion\App\Domain\Entity\Chart\Pie\PieChart</discriminator-class>
        <discriminator-groups>
            <group>index</group>
            <group>detail</group>
        </discriminator-groups>
        <property name="dtype" exclude="true"/>
    </class>
</serializer>

<?php
class ChartService
{
    protected $em, $account;

    public function __construct(\Doctrine\ORM\EntityManager $em, \NotionCommotion\App\Domain\Entity\Account\Account $account) {
        $this->em = $em;
        $this->account = $account;
    }

    public function read(int $idPublic):Chart\Chart {
        return $this->em->getRepository(Chart\Chart::class)->findOneBy(['id_public'=>$idPublic, 'account'=>$this->account]);
    }
}

<?php
class ChartResponderr
{
    protected $serializer;

    public function __construct(\JMS\Serializer\Serializer $serializer) {
        $this->serializer = $serializer;
    }

    public function detail(Response $response, Entity\object $entity):Response
    {
        $json=$this->serializer->serialize($entity, 'json', SerializationContext::create()->setGroups(['Default','detail']));
        $response->getBody()->write($json);
        return $response;
    }
}

<?php
$c['serializer'] = function ($c) {
    return \JMS\Serializer\SerializerBuilder::create()
    ->setCacheDir(APP_ROOT.'/var/serializer/cache')
    ->setDebug(true)
    ->addMetadataDir(APP_ROOT.'/config/serializer')
    ->build();
};

$c['chartService'] = function ($c) {
    return new Api\Chart\ChartService(
        $c[EntityManager::class],
        $c['account']
        //$c['validator'](['/chart/base.json'])
    );
};

$app->get('/chart/{chartId:[0-9]+}', function (Request $request, Response $response, $args) {
    return $this->chartResponder->detail($response, $this->chartService->read((int) $args['chartId']));
});