how can we fetch the JSON - Data from a page

dil_bert · February 20, 2018

dear PHP-Freaks

The French government has created a website showing the list of all 4307 hospitals and clinics in France: http://www.ScopeSante.fr
We can see the list of all establishments web page, The site uses its own private API to get the list of establishments in the first place.

Check out: https://ws.scopesante.fr/v4/carte-etablissements

https://ws.scopesante.fr/v4/carte-etablissements

0    {…}
1    
n    "CENTRE HOSPITALIER DE BELLEY"
id    "010000032"
cert    {…}
s    1
t    2
geo    […]
2    
n    "CH DE TREVOUX - MONTPENSIER"
id    "010000065"
cert    {…}
s    1
t    2
geo    […]
3    {…}
4    
n    "CENTRE HOSPITALIER DE MEXIMIEUX"
id    "010000099"

How can we get the IDs from the page

requinix · February 20, 2018

By fetching the contents of the page and decoding with json_decode(). As as with every JSON API.

Can you be more precise about what you don't know how to do?

dil_bert · February 21, 2018

hello dear requinix many thanks for the quick reply.

what is aimed; i want to have a full list of the dataset.

see the https://ws.scopesante.fr/v4/carte-etablissements

The private API returns the full list of establishments along with their "id" numbers in JSON format.This is a great approach; This is the full list. We can use the "id" number of each organization to look up the "details" page for each organization. So, using the route to the "details" page: https://www.scopesante.fr#/etablissements/PUT_ID_NUMBER_HERE/fiche-detaillee/

We can get the details for a specific establishment directly by calling the API. See:
https://ws.scopesante.fr/v4/fiche-identite/010000024

This returns the details on the specific establishment.
We can replace the number on the end of the URL with the "id" number of the establishment you are looking up. Using an API is usually more efficient than scraping the pages directly.

we have for example the follwing ending of a URL

010000024

this number is also visible in the overview see the https://ws.scopesante.fr/v4/carte-etablissements

so i have to do some combination:

the question is - how to retrieve the full dataset of the approx 4000 records ?

i guess that the key is

a. retrival

b. processing of JSON-data and

c. creating a db that stores the whole dataset

now i want to have a closer look at the decoding with json_decode() .- as with every JSON API.

many thanks for your help here.

dil_bert · February 21, 2018

Hello dear requinix,

the goal - what is aimed: i need to have the data of the etablissements that are located in the following two towns:

1, Rennes and

2. Avrances - these are fairly little towns in Brittany.a a part of France. .

I guess that we re talking about 20 to 30 records - out of an huge amount of approx 4000 records that are stored in the database.

In fact : i only need to have the etablissements of those both towns, Avranches and Rennes.

Method: I try to achieve this by at the looking at decoding with json_decode() .- as with every JSON API.

and then - try to get the IDs and then try to figure out if i can go ahead with this.

perhaps i filter out the dataset - for the above mentioned towns

1. Rennes

2. Avranches

Example: The dataset - looks in general - like so; (see the links i presented above)

[{"n" : "CENTRE HOSPITALIER DE FLEYRIAT", "id" : "010000024", "cert" :{"cot" : "B", "clr" : 2, "v" :4 }, "s" : 1, "t" : 3, "geo" : [5.228133,46.204833] },{"n" : "CENTRE HOSPITALIER DE BELLEY", "id" : "010000032", "cert" :{"cot" : "B"

, "clr" : 2, "v" :0 }, "s" : 1, "t" : 2, "geo" : [5.684976,45.761208] },{"n" : "CH DE TREVOUX - MONTPENSIER", "id" : "010000065", "cert" :{"cot" : "A", "clr" : 1, "v" :4 }, "s" : 1, "t" : 2, "geo" : [4.772192,45.940741] },{"n" : "CENTRE HOSPITALIER DU PAYS DE GEX", "id" : "010000081", "cert" :{"cot" : "B", "clr" : 2, "v" :0 }, "s" : 1, "t" : 2, "geo" : [6.057731,46.322337] },{"n" : "CENTRE HOSPITALIER DE MEXIMIEUX", "id" : "010000099", "cert" :{"cot" : "B", "clr" : 2, "v" :4 }, "s" : 1, "t" : 2, "geo" : [5.194579,45.906191] },{"n" : "CENTRE HOSPITALIER DE PONT DE VAUX", "id" : "010000107", "cert" :{"cot" : "B", "clr" : 2, "v" :0 }, "s" : 1, "t" : 2, "geo" : [4.947451,46.431885] },{"n" : "CHI AIN VAL DE SAONE - PONT VEYLE", "id" : "010000115", "cert" :{"cot" : "B", "clr" : 2, "v" :4 }, "s" : 1, "t" : 2, "geo" : [4.890054,46.263472] },{"n" : "CHI AIN VAL DE SAONE - THOISSEY", "id" : "010000131", "cert" :{"cot" : "B", "clr" : 2, "v" :4 }, "s" : 1, "t" : 2, "geo" : [4.804417,46.172133] },{"n" : "CH PUBLIC HAUTEVILLE - UNITE ESPERANCE", "id" : "010000180", "cert" :{"cot" : "A", "clr" : 1, "v" :4 }, "s" : 1, "t" : 2, "geo" : [5.599563,45.978899] },{"n" : "CH PUBLIC HAUTEVILLE - UNITE INTERDEPT", "id" : "010000198", "cert" :{"cot" : "A", "clr" : 1, "v" :4 }, "s" : 1, "t" : 2, "geo" : [5.599563,45.978899] },{"n" : "CH PUBLIC HAUTEVILLE - UNITE ALBARINE", "id" : "010000214", "cert" :{"cot" : "A", "clr" : 1, "v" :4 }, "s" : 1, "t" : 2, "geo" : [5.599563,45.978899] },{"n" : "CENTRE PSYCHOTHERAPIQUE DE L'AIN", "id" : "010000495", "cert" :{"cot" : "B", "clr" : 2, "v" :4 }, "s" : 3, "t" : 3, "geo" : [5.226838,46.216428] },{"n" : "CLINIQUE DE READAPTATION LES ARBELLES", "id" : "010002129", "cert" :{"cot" : "B", "clr" : 2, "v" :0 }, "s" : 2, "t" : 2, "geo" : [5.231909,46.194207] },{"n" : "CH DU HAUT BUGEY - GEOVREISSET", "id" : "010005239", "cert" :{"cot" : "C", "clr" : 3, "v" :4 }, "s" : 1, "t" : 2, "geo" : [5.607031,45.727813] },{"n" : "HAD DE L'HOPITAL PRIVE D'AMBERIEU", "id" : "010005379", "cert" :{"cot" : "B", "clr" : 2, "v" :0 }, "s" : 2, "t" : 2, "geo" : [5.357838,45.958143] },{"n" : "AURAL UNIT\u00c9 DIALYSE CH HAUT BUGEY", "id" : "010006526", "cert" :{"cot" : "B", "clr" : 2, "v" :4 }, "s" : 3, "t" : 1, "geo" : [5.630206,46.259835] },{"n" : "CLINIQUE AMBULATOIRE CENDANEG", "id" : "010007300", "cert" :{"cot" : "C", "clr" : 3, "v" :4 }, "s" : 2, "t" : 2, "geo" : [6.086062,46.248454] },{"n" : "SSR HOTEL DIEU DE FLEYRIAT", "id" : "010009314", "cert" :{"cot" : "B", "clr" : 2, "v" :4 }, "s" : 1, "t" : 2, "geo" : [5.233912,46.200719] },{"n" : "USLD RESIDENCE EMILE PELICAND", "id" : "010009702", "cert" :{"cot" : "B", "clr" : 2, "v" :4 }, "s" : 1, "t" : 2, "geo" : [5.2201528,46.2041143] },{"n" : "CLINIQUE CONVERT", "id" : "010780195", "cert" :{"cot" : "A", "clr" : 1, "v" :4 }, "s" : 2, "t" : 2, "geo" : [5.253603,46.211525] },{"n" : "HOPITAL PRIVE D'AMBERIEU", "id" : "010780203", "cert" :{"cot" : "B", "clr" : 2, "v" :0 }, "s" : 2, "t" : 2, "geo" :

Many thanks for the support !!

greetings Dil_bert

Edited February 21, 2018 by dil_bert

requinix · February 21, 2018

I'm not exactly sure if they support people doing this sort of thing... but the API looks actually quite public, with how it's designed...

You can get the JSON from the first page. Use json_decode to turn the JSON array into PHP array/object. You can then loop over all the IDs in the data and get the JSON from each page and decide what you want to do from there.

Look into json_decode(), write some code, try it out, and if you have problems post what you're written and what problem you have.

dil_bert · February 21, 2018

hello dear requinix,

many thanks for the reply -

I'm not exactly sure if they support people doing this sort of thing... but the API looks actually quite public, with how it's designed...

i am only looking for a little (very little) amount of data.

in fact all i am interested in is the retrival of data for two towns -

a. Avranches

b. Rennes

i suppose that this set of data contains approximatly 30 to 40 records.

I will use the method you described. If i run into any issues i come back and report.

greetings

Barand · February 21, 2018

There are 6 records containing the word "Rennes" and 3 with "Avranches". How do plan on recognising the 30-40 records from those regions?

dil_bert · February 21, 2018

Hello dear Barand,

many thanks - regarding the records that contain data from the intended area: well my ideas from above - those numbers were only first and preliminary estimations.

one of the main goals is to get some kind of experience to work with such a API and to get to know how to handle the processes. I guess that the first step is

to fetch the ID - then to get an overview on all the entries and finally to do some concrete and direct search after the keywords

- Avranches

- Rennes

So - for me this is also a great options to learn.

Learn to approach and to find a way in coding, parsing, and fetching data in general. And additonally: to store JSON-Data.

To be frank: i am pretty new to the process of decoding with json_decode(). But i will dive into it - i can learn alot about the work with general JSON API.

I can benefit from this little project - and learn to find a way to go through such tasks

Many thanks dear requinix and Brand - for the comments and the idea-sharing

greetings

Edited February 21, 2018 by dil_bert

Barand · February 21, 2018

The only way I could see to get the data for those areas (given the establishments had lat and long data) was to search for those near those towns. Setting a search radius of 30Km gives 27 for the Rennes area and 13 for the Avranches area.

This is the code I used

#------------------------------------------#
#   Function to get the json data          #
#------------------------------------------#
function get_data($path){
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_URL,$path);
    curl_setopt($ch, CURLOPT_FAILONERROR,1);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION,1);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
    curl_setopt($ch, CURLOPT_TIMEOUT, 15);
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    $retValue = curl_exec($ch);
    curl_close($ch);
    return $retValue;
}

#------------------------------------------#
#   Functions to filter the data for those #
#   places within 30Km of the two area     #
#------------------------------------------#
function ren_filter($rec)
{
    $rennes = [48.111883, -1.682277];
    $maxdistance = 30; //(km)
    list($lat1, $long1) = $rennes;
    list($long2, $lat2) = $rec['geo'];
    return sqrt(pow(($lat1-$lat2),2) + pow(($long1-$long2),2))*111 <= $maxdistance;
}

function avr_filter($rec)
{
    $avranches = [48.6891341,-1.3739595];
    $maxdistance = 30; //(km)
    list($lat1, $long1) = $avranches;
    list($long2, $lat2) = $rec['geo'];
    return sqrt(pow(($lat1-$lat2),2) + pow(($long1-$long2),2))*111 <= $maxdistance;
}

#---------------------------------------------#
#   Process the data                          #
#---------------------------------------------#
$url1 = 'https://ws.scopesante.fr/v4/carte-etablissements';

$data = json_decode(get_data($url1), 1);                 // get the data

$rennes_data    = array_filter($data, 'ren_filter');     // filter out the Rennes area data
$avranches_data = array_filter($data, 'avr_filter');     // filter out the Avranches area data

#----------------------------------------------#
#   Output the two sets of data                #
#----------------------------------------------#
echo '<h3>Rennes</h3><pre>';
printf('%-10s| %-45s|%8s<br><br>', 'Id', 'Name', 'Dist');
foreach ($rennes_data as $rec) {
    $rennes = [48.111883, -1.682277];
    list($lat1, $long1) = $rennes;
    list($long2, $lat2) = $rec['geo'];
    $dist = sqrt(pow(($lat1-$lat2),2) + pow(($long1-$long2),2))*111;
    printf('%-10d| %-45s|%8.2f<br>', $rec['id'], $rec['n'], $dist) ;
}

echo '</pre><h3>Avranches</h3><pre>';
printf('%-10s| %-45s|%8s<br><br>', 'Id', 'Name', 'Dist');
foreach ($avranches_data as $rec) {
    $avranches = [48.6891341,-1.3739595];
    list($lat1, $long1) = $avranches;
    list($long2, $lat2) = $rec['geo'];
    $dist = sqrt(pow(($lat1-$lat2),2) + pow(($long1-$long2),2))*111;
    printf('%-10d| %-45s|%8.2f<br>', $rec['id'], $rec['n'], $dist) ;
}
echo '</pre>';

and these are the results

	Rennes
Id | Name | Dist

350000014 | CHU/ CHRU RENNES SITE HOTEL DIEU | 0.69
350000121 | CHP SAINT GREGOIRE | 3.72
350000139 | CLINIQUE MUTUALISTE DE LA SAGESSE | 3.49
350000337 | C.H. GUILLAUME REGNIER RENNES | 3.15
350000410 | CH DE LA ROCHE AUX FÉES JANZE  | 26.31
350000741 | CHU/ CHRU RENNES SITE PONTCHAILLOU | 1.75
350002119 | CLINIQUE DU MOULIN | 11.24
350002176 | CLINIQUE DE L'ESPERANCE | 0.84
350002192 | POLYCLINIQUE ST LAURENT | 3.99
350002200 | CLINIQUE SAINT YVES | 2.55
350002234 | CENTRE MEDICAL ET PEDAGOGIQUE BEAULIEU | 5.10
350002564 | POLE MPR SAINT HELIER | 1.52
350002747 | CTRE SSRAA L'ESCALE | 2.27
350002754 | CENTRE LA THEBAUDAIS | 3.09
350002804 | CENTRE DE DIALYSE AMBULATOIRE AUB | 3.54
350002812 | C.R.L.C.C. EUGENE MARQUIS | 2.43
350005021 | POLE GERIATRIQUE RENNAIS | 7.96
350005146 | HOPITAL PRIVE SEVIGNE | 6.49
350005278 | CENTRE REY LEROUX | 25.80
350006771 | CHU/ CHRU RENNES SITE LA TAUVRAIS | 3.70
350007084 | CHU/ CHRU RENNES SITE HOPITAL SUD | 3.92
350008579 | URFMP LE PATIS FRAUX | 11.76
350032280 | RESIDENCE DU TERTRE DE JOUE | 2.89
350032934 | UNITE DIALYSE PONTCHAILLOU RENNES AUB | 1.46
350042628 | HAD 35 | 0.26
350044756 | CLINIQUE PHILAE | 12.46
350046751 | UNITE DIALYSE MONTGERMONT AUB | 5.62

	Avranches
Id | Name | Dist

350000444 | CH DES MARCHES DE BRETAGNE - ANTRAIN | 28.38
500000021 | CHAG - SITE AVRANCHES | 28.85
500000146 | POLYCLINIQUE DE LA BAIE-ST MARTIN | 2.75
500000229 | CRF LE NORMANDY - GRANVILLE | 29.91
500000260 | CHAG - SITE GRANVILLE | 26.52
500000435 | HOPITAL SAINT JAMES | 18.97
500000476 | HOPITAL LOCAL - VILLEDIEU LES POELES | 23.90
500000575 | CENTRE HOSPITALIER DE L'ESTRAN | 22.04
500013081 | USLD - CH AVRANCHES-GRANVILLE | 26.52
500021282 | HAD DU CH DE VIRE SITE DE VILLEDIEU | 23.93
500021316 | CENTRE DE DIALYSE D'AVRANCHES | 2.67
500021415 | ANIDER - SARTILLY | 11.92
500021423 | CRF LE NORMANDY II | 26.51

Edited February 21, 2018 by Barand

dil_bert · February 21, 2018

dear Barand,

Wow - many many thanks - this was more than expected; I am overwhelmed by such a amazing help and support. You have helped my several time so far.

This is a great approach and i am really excited.

Again many many thanks for all you did! You are encouraging me to go ahead. with my plan to learn more about coding and processing data.

Above all: I am so glad to be here - in this great place!!!

Greetings Dil_bert

Edited February 21, 2018 by dil_bert

Sign In

how can we fetch the JSON - Data from a page

Recommended Posts

dil_bert

Link to comment

Share on other sites

requinix

Link to comment

Share on other sites

dil_bert

Link to comment

Share on other sites

dil_bert

Link to comment

Share on other sites

requinix

Link to comment

Share on other sites

dil_bert

Link to comment

Share on other sites

Barand

Link to comment

Share on other sites

dil_bert

Link to comment

Share on other sites

Barand

Link to comment

Share on other sites

dil_bert

Link to comment

Share on other sites

Join the conversation

Browse

Activity

Important Information