Advise on how to start this

[oracle765]

There are 2 feeds. Format: tab-separated values.

A list of retailers
http://www.getprice.com.au/affinityclick/ShopsList.aspx?Pass=1iob2kjd&ref=cc

A list of products ( example for ID 50 -- Bevilles ) http://www.getprice.com.au/affinityclick/shop_products_list.aspx?id=50&Pass=1iob2kjd&ref=cc


Shop list
•	id -- internal retailer id
•	name -- retailer's name
•	url -- retailer's website url
•	logo -- retailer's logo. Example: http://www.getprice.com.au/images/shoplogo/rebelsport.gif
•	rank -- rating from 0 to 5
•	products List -- the url to the list of products from the retailer.

Product list
•	Shop_id -- internal retailer id
•	offer name -- example: Bburago 1/18 Gold Collection Diecast Car: Mini Cooper (1969) (Green/White)
•	model -- example: SMS-840243
•	image -- image url. Example: http://www.getprice.com.au/images/uploadimg/66/_1_SMS-840243-1.jpg
•	price -- example: 44.99
•	sale_price -- example: 44.99
•	brand -- example: Mini Cooper
•	link_URL -- product url. Example: http://origin.getprice.com.au/prodhits.aspx?pid=67072476&shopid=66&ref=superfish&refname=cc
•	category_name -- example: Vehicles & Planes
•	Sort_rank -- example: 98
•	Unique_GetpriceID -- example: 67072476

Hi professionals

 

I have been given access to feeds for our comparison site and I am not sure where to start with regards to importing the data into the database and also how I would go about querying the dataset once its in there I can set up a cron job to import to folder no problem but I am confused with the dataset.  I am guessing that it would need to import all products and not just the example product feed for id 50.

 

A working example can be found on yahoo shopping.yahoo.com.au in their search box you can pretty much type in anything and it will search for all keywords.

 

What column out of the feeds would it query Eg would it do something like

 

select col1,col2,col2 from table

where keyword in (?)

 

I am a bit confused in the best way to approach this.

 

Please find attached the feed doc I have to work with and I hope I am making enough sense

 

 

thanks in advance

[oracle765]

SCRIPT

#Get price retailers list
/usr/bin/wget -O "/home/oracle765/public_html/feed/getprice.txt" 'http://www.getprice.com.au/affinityclick/ShopsList.aspx?Pass=1iob2kjd&ref=cc';


use DBI;

my $dbh = DBI->connect("DBI:mysql:database=mydatabase;host=localhost", "USERNAME", 'PASSWORD', {'RaiseError' => 1});

$SQL = "LOAD DATA INFILE '$datafile' REPLACE INTO TABLE getpriceretailer FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n' (id,name,url,logo,rank,products,List)";

$dbh->prepare($SQL);
$dbh->execute;
$dbh->finish;
$dbh->disconnect;




ERROR IN EMAIL



HTTP request sent, awaiting response... 200 OK
Length: 244130 (238K) [text/plain]
Saving to: “/home/oracle765/public_html/feed/getprice.txt”

     0K .......... .......... .......... .......... .......... 20% 1.52M 0s
    50K .......... .......... .......... .......... .......... 41% 1.71M 0s
   100K .......... .......... .......... .......... .......... 62%  907K 0s
   150K .......... .......... .......... .......... .......... 83% 1.13M 0s
   200K .......... .......... .......... ........             100%  919K=0.2s

2014-03-31 23:04:02 (1.16 MB/s) - “/home/oracle765/public_html/feed/getprice.txt” saved [244130/244130]

/home/oracle765/public_html/scripts/feedimport.sh: line 14: use: command not found
/home/oracle765/public_html/scripts/feedimport.sh: line 16: syntax error near unexpected token `('
/home/oracle765/public_html/scripts/feedimport.sh: line 16: `my $dbh = DBI->connect("DBI:mysql:database=mydatabase;host=localhost", "USERNAME", 'PASSWORD', {'RaiseError' => 1});'

Hi All

 

I have tried this in a cronjob with a file called feeds.sh as I read something on a forum, but I am getting this error

[oracle765]

#Get price retailers list
/usr/bin/wget -O "/home/oracle765/public_html/feed/getprice.txt" 'http://www.getprice.com.au/affinityclick/ShopsList.aspx?Pass=1iob2kjd&ref=cc';


$dbh = mysql -u USERNAME -p PASSWORD -h mysql.compareandchoose.com.au mydatabase # < /home/oracle765/public_html/feed/getprice.txt

$SQL = "LOAD DATA INFILE '/home/oracle765/public_html/feed/getprice.txt' REPLACE INTO TABLE getpriceretailer FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\n' (id,name,url,logo,rank,products,List)";

$dbh->prepare($SQL);
$dbh->execute;
$dbh->finish;
$dbh->disconnect;



AND IT SHOWS IN MY EMAIL

2014-04-01 00:29:02 (1.14 MB/s) - “/home/oracle765/public_html/feed/getprice.txt” saved [244130/244130]

/home/oracle765/public_html/scripts/feedimport.sh: line 14: =: command not found
/home/oracle765/public_html/scripts/feedimport.sh: line 16: =: command not found
/home/oracle765/public_html/scripts/feedimport.sh: line 18: syntax error near unexpected token `('
/home/oracle765/public_html/scripts/feedimport.sh: line 18: `$dbh->prepare($SQL);'

got this far now