Unzip script

[dfowler]

Hey guys, I have a tough on here.  The process will start with an SQL dump.  The dump contains several zip files and the dump is scheduled once per week.  Upon downloading the files, the script should unzip them and update a table in a remote mysql server with the records from 3 separate zip files. . . then place images on a known location in a linux server.

 

This is way over my head, and I have no idea how to accomplish this at all.  Help would be GREATLY appreciated.  Let me know if you would need any other info as I'm still trying to figure things out myself. 

[dfowler]

I think the problem for me will be unzipping the files and reading them.  Once I get the information I'm pretty confident I can update the table in the database.

[rhodesa]

You don't even need to extract the ZIP, you can read files from it directly with ZipArchive:

 

http://us2.php.net/manual/en/function.ziparchive-getfromname.php

[dfowler]

Oh that is great!  So I'm assuming I can do something like the following:

<?php
$zip = new ZipArchive;
if ($zip->open('test1.zip') === TRUE) {
    $testdoc1 = $zip->getFromName('testfromfile.txt');
    $zip->close();
} else {
    echo 'failed';
}
?>

 

This will place all the information from that file into $testdoc1?  Here is an example of what a line of information from the file will look like:

2.44|||0|||||0|||||||||||||||||||||||Tinsel County||||||||||||||||||Smallville|None|||||||||||||||||||||||||||||||||||||||||||None||||||||||||||||||||||Y|Yes|||||||||||||||419000|LOVETTDE|413|363 x 334 x 381 x 251|2-3 Acres|||||||||||||||||Cash|Conventional||||||||||79385|||||||||Commercial||||||||||||Unimproved||Commercial|Acreage/Farm|Property to be sold consists of 2.44 acres and is zoned c|ommerical - c1 - Approximately 363 road frontage on high |traffic street.  Lot's of new residential and commercial |activities in this area.||||||||||||||||||||||||||||None||||||||||||Crowsnest|0|Road||||||||||||||None||||No||||||None|||||||31405|||||||

 

I am going to have to place this into a database, so I'm assuming I will need to add an explode to get the correct information.

[rhodesa]

Correct, the contents of the text file should be stored in $testdoc1 after that.

[dfowler]

Ok, the continuing struggles of this project.  After I get the information from the zip, each property it is separated by a carriage return chr(10) or chr(13) (or both most likely)  and each feature is separated by  a |  (pipe).  So we have a (carriage return) delimited list of pipe delimited features.

 

I need a script that will loop through the list of properties, then within each property, loop through the list of features.  Then, with some planning, I need to pluck a few items off that list of features.  Those items I need to add to db, then move to the next property and do the same.

 

It also appears that the ZIP and files within the ZIP will have a different name each time.  For example, 1st ZIP - property2111, next it will be property2443, property3421, etc....

 

The files within will be something like 1st ZIP file - 132232-RES, then 4353434-RES, 2342123234-RES, etc....

 

How do I pull this one off.  I obviously am way over my head here, and I would appreciate any help I can get!

[rhodesa]

Wow, um, you may want to take this over to the freelance area: http://www.phpfreaks.com/forums/index.php/board,8.0.html

 

You can also send me a PM and I might be able to work something out with you.

[redarrow]

All depends how you want the info in the database mate...

 

SHOW ME WITH THE CURRENT POSTED CODE I HAVE PROVIDED HOW YOU

WANT THE DATA TO LOOK IN THE DATABASE

THEN I CAN HELP...

 

DESCIBE THE DATABASE FIELDS AND THE INFO NEEDED FOR THE INFO PLEASE

 

example only...

 

<?php

$test="2.44|||0|||||0||||||||||||||||||||||
|Tinsel County||||||||||||||||||Smallville|None||||||
|||||||||||||||||||||||||||||||||||||None||||||||||||||||||||||Y|Yes
|||||||||||||||419000|LOVETTDE|413|363 x 334 x 381 x 251|2-3 Acres|||||||
||||||||||Cash|Conventional||||||||||79385|||||||||Commercial||||||||||||Unim
proved||Commercial|Acreage/Farm|Property to be sold consists of 2.44 acres and is 
zoned c|ommerical - c1 - Approximately 363 road frontage on high |traffic street.  Lot'
s of new residential and commercial |activities in this area.|||||||||||||||||||||||||||
|None||||||
||||||Crowsnest|0|Road||||||||||||||None||||No||||||None||
|||||31405|||||||";

$test=explode(' ',$test);

$test=str_replace('|','',$test);

$test=implode('<br>',$test);

echo "$test";

 

array format change array numbers as needed..

 

<?php

$test="2.44|||0|||||0||||||||||||||||||||||
|Tinsel County||||||||||||||||||Smallville|None||||||
|||||||||||||||||||||||||||||||||||||None||||||||||||||||||||||Y|Yes
|||||||||||||||419000|LOVETTDE|413|363 x 334 x 381 x 251|2-3 Acres|||||||
||||||||||Cash|Conventional||||||||||79385|||||||||Commercial||||||||||||Unim
proved||Commercial|Acreage/Farm|Property to be sold consists of 2.44 acres and is 
zoned c|ommerical - c1 - Approximately 363 road frontage on high |traffic street.  Lot'
s of new residential and commercial |activities in this area.|||||||||||||||||||||||||||
|None||||||
||||||Crowsnest|0|Road||||||||||||||None||||No||||||None||
|||||31405|||||||";

$test=explode(' ',$test);


$test=str_replace('|','',$test);

print_r($test);

$test=implode('<br>',$test);

echo "$test";

?>

 

result

X-Powered-By: PHP/5.2.0
Content-type: text/html

Array
(
    [0] => 2.4400
Tinsel
    [1] => CountySmallvilleNone
NoneYYes
419000LOVETTDE413363
    [2] => x
    [3] => 334
    [4] => x
    [5] => 381
    [6] => x
    [7] => 2512-3
    [8] => Acres
CashConventional79385CommercialUnim
provedCommercialAcreage/FarmProperty
    [9] => to
    [10] => be
    [11] => sold
    [12] => consists
    [13] => of
    [14] => 2.44
    [15] => acres
    [16] => and
    [17] => is
    [18] => 
zoned
    [19] => commerical
    [20] => -
    [21] => c1
    [22] => -
    [23] => Approximately
    [24] => 363
    [25] => road
    [26] => frontage
    [27] => on
    [28] => high
    [29] => traffic
    [30] => street.
    [31] => 
    [32] => Lot'
s
    [33] => of
    [34] => new
    [35] => residential
    [36] => and
    [37] => commercial
    [38] => activities
    [39] => in
    [40] => this
    [41] => area.
None
Crowsnest0RoadNoneNoNone
31405
)
2.4400
Tinsel<br>CountySmallvilleNone
NoneYYes
419000LOVETTDE413363<br>x<br>334<br>x<br>381<br>x<br>2512-3<br>Acres
CashConventional79385CommercialUnim
provedCommercialAcreage/FarmProperty<br>to<br>be<br>sold<br>consists<br>of<br>2.44<br>acres<br>and<br>is<br>
zoned<br>commerical<br>-<br>c1<br>-<br>Approximately<br>363<br>road<br>frontage<br>on<br>high<br>traffic<br>street.<br><br>Lot'
s<br>of<br>new<br>residential<br>and<br>commercial<br>activities<br>in<br>this<br>area.
None
Crowsnest0RoadNoneNoNone
31405

 

 

IT TRUE TO SAY SOME USERS WILL CHARGE YOU BUT MOST PHP

PROGRAMMERS LEARNING ALWAYS WILL DO IT FOR FREE FOR FUN AND EXSPERINCE..

[redarrow]

Let say the first 0 array,

was the name off the data then inset into the datbase as name.

 

 

 

<?php

$test="2.44|||0|||||0||||||||||||||||||||||
|Tinsel County||||||||||||||||||Smallville|None||||||
|||||||||||||||||||||||||||||||||||||None||||||||||||||||||||||Y|Yes
|||||||||||||||419000|LOVETTDE|413|363 x 334 x 381 x 251|2-3 Acres|||||||
||||||||||Cash|Conventional||||||||||79385|||||||||Commercial||||||||||||Unim
proved||Commercial|Acreage/Farm|Property to be sold consists of 2.44 acres and is 
zoned c|ommerical - c1 - Approximately 363 road frontage on high |traffic street.  Lot'
s of new residential and commercial |activities in this area.|||||||||||||||||||||||||||
|None||||||
||||||Crowsnest|0|Road||||||||||||||None||||No||||||None||
|||||31405|||||||";

$test=explode(' ',$test);


$test=str_replace('|','',$test);

for($i=0; $i<count($test); $i++){

$name=$test[0][$i];

//insert to database the name......
}
?>

[dfowler]

The database will be a list of all the properties and their features.  Basically what I am doing is take a local real estate website I've created, and updating it's properties based off of the local MLS.  The zip is the dump from the MLS, so I am trying to take the information from it to update the local real estate agency site.  The fields are as follow:

 

# Bedrooms : NUMBER[3]

# Building Stories : NUMBER[4]

# Full Baths : NUMBER[2]

# Half Baths : NUMBER[2]

# of Images : NUMBER[2]

# Unit Stories : NUMBER[4]

Address : CHARACTER[100]

Advertised Subdivision : CHARACTER[30]

Area : NUMBER[3]

City : CHARACTER[30]

Internet-Address Allowed? : CHARACTER[1]

Internet Listing Display : CHARACTER[1]

Lease Info : CHARACTER[x]

Lease Term Ends : CHARACTER[5]

List Price : NUMBER[9]

Listing Office : CHARACTER[12]

Lot Size : CHARACTER[9]

Mls# : CHARACTER[10]

Property Subtype : CHARACTER[8]

Public Remarks : CHARACTER[X]

Square Feet : NUMBER[6]

Style : CHARACTER[X]

Unit # : CHARACTER[4]

Unit Floor : NUMBER[2]

Year Built : NUMBER[5]

ZipCode : CHARACTER[12]

Listing Agent Public Id : CHARACTER[15]

State : CHARACTER[2]

[redarrow]

someone please help going out now, sorry try to solve later....

[dfowler]

Well, I'm plugging away at this.  Here is my question, I can use the

<?php
$zip = new ZipArchive;
if ($zip->open('test1.zip') === TRUE) {
    $testdoc1 = $zip->getFromName('testfromfile.txt');
    $zip->close();
} else {
    echo 'failed';
}
?>

that rhodesa showed me.  However, how can I get the file?  In SQL I would just use something

SELECT WHERE name LIKE '%property%'

However, I don't know how that would work with this PHP function.  Is there a way to get a file if you only know it's partial name?

As I said before 1st ZIP will be something similar to property2111, next it will be property2443, property3421, etc.... 

 

Thanks for any information!

 

[rhodesa]

Here is a list of all the ZipArchive functions: http://us3.php.net/manual/en/ref.zip.php

 

Off the top of my head, you would probably use getNameIndex() to loop through each item in the archive. Test the value from the function to see if it matches, and open it if it does.

[dfowler]

What about for the zip itself?  It is going to have a different name each time as well as the contents.  Also, I'm assuming you mean something like this for the file within the zip:

 

<?php
$i=0;
$zip = new ZipArchive;
if ($zip->open('test1.zip') === TRUE) {
while($zip->getNamefromIndex($i) != false) {
	$testdoc1 = $zip->getNamefromIndex($i);
	if (eregi('RES', $testdoc1)) {
		$res = $zip->getFromName($testdoc1);
		$zip->close();
	}
	$i++
} else {
    echo 'failed';
}
?>

[dfowler]

Ok, I think I found something I could use for the zip itself.  It is very similar to what I was using before:

<?php
$ftp_files = array();
$ftp_conn = ftp_connect(FTP_HOST);
ftp_login($ftp_conn,FTP_USER,FTP_PASS);
$ftp_files = ftp_nlist($ftp_conn, ".");

foreach ($ftp_files as $f){
if (eregi('property', $f)) {
	$zip_name = $f;
}
}

$i=0;
$zip = new ZipArchive;
if ($zip->open($zip_name) === TRUE) {
while($zip->getNamefromIndex($i) != false) {
	$testdoc1 = $zip->getNamefromIndex($i);
	if (eregi('RES', $testdoc1)) {
		$res = $zip->getFromName($testdoc1);
		$zip->close();
	}
	$i++
}
} else {
    echo 'failed';
}

ftp_close($ftp_conn);
?>

[dfowler]

Hey guys, here is where I am now:

<?php

// {{{ constants
define('FTP_HOST','url');
define('FTP_USER','user');
define('FTP_PASS','pass');
$ftp_files = array();
$image = array();
//}}}


set_time_limit(3600);

//{{{ get file from ftp
$ftp_conn = ftp_connect(FTP_HOST);
ftp_login($ftp_conn,FTP_USER,FTP_PASS);
$ftp_files = ftp_nlist($ftp_conn, ".");

foreach ($ftp_files as $f){
if (eregi('RES', $f)) {
	$zip_name = $f;
}
}

$i=0;
$zip = new ZipArchive;
if ($zip->open($zip_name) === TRUE) {
while($zip->getNamefromIndex($i) != false) {
	$testdoc1 = $zip->getNamefromIndex($i);
	if (eregi('RES', $testdoc1)) {
		$res = $zip->getFromName($testdoc1);
		$zip->close();
	} else if (eregi('.jpg', $testdoc1)) {
		$image[] = $zip->getFromName($testdoc1);
		$zip->close();
	$i++
}
} else {
    echo 'failed';
}

foreach ($image as $p){
function ftp_copy($p) {
	$ftp_root = '/public_html/images/';
	$site_root = '/home/usr/public_html/images/';
	return ftp_put($ftp_conn, $ftp_root . $p, $site_root . $p, FTP_BINARY);
}
}

ftp_close($ftp_conn);
//}}}


mysql_close($conn);
?>

 

My question now is, should I just extract the zip and get the info from there?  This would make the moving of the images a lot easier I would think.  Also, I still haven't tackled the actual database property stuff just yet.