Php code to read files directly

[dev1902]

Hello, right now i'm running a PHP code that looks into a directory and retrieves the files.
--Does anyone know a simple, quick and efficientPHP code format to go straight to the files in a directory instead of going through the directory itself?

Right now i'm running this code. 

php code:

 <?php
                    if( is_dir( 'temp/' ) ) {
                                $dHandle = opendir( "temp/" );  
                        while( false !== ($entry = readdir($dHandle) ) ) {  
                                if( preg_match( "/(.+).gfr$/", $entry, $matches) > 0 ) {  
                                                                $sample = $matches[1];
 
?>
 

--I need a code to go straight to the files so it could process faster on my web database text form. 
--FYI, i have over 25,00 files in this folder.

 

 

[AbraCadaver]

glob() is great.

$files = glob("temp/*.gfr"); //for array of files

//or

foreach(glob("temp/*.gfr") as $file) {
   //do something with $file
}

[QuickOldCar]

If you know the filename you want can do something like this, you can add your preg match checking to it.

 

 

<?php
//read a file
$my_file = "filename.txt";
if (file_exists($my_file)) {
    $data  = file($my_file);
    $total = count($data);
    echo "<br />Total lines: $total<br />";
    
    
    foreach ($data as $line) {
        $line = trim($line);
        echo "$line<br />";
    }
    
} else {
    echo "No file to display";
}
?>  

[dev1902]

This is one the the full codes i actually run: 

<form action="resultsSummary.php" method="post" name="sampleDetail" target="_self" class="container" AUTOCOMPLETE="off">

            

 

<form action="resultsSummary.php" method="post" name="sampleDetail" target="_self" class="container" AUTOCOMPLETE="off">
            
		  <h1 >           
          	
            <label for="sampleID">Gene 1: </label>
            <input name='sampleID' id='sampleID' list="gene1">
            <datalist id="gene1">
		      <?php
              	    if( is_dir( 'GFR/' ) ) {
	                	$dHandle = opendir( "GFR/" );  
                    	while( false !== ($entry = readdir($dHandle) ) ) {  
                    		if( preg_match( "/(.+).gfr$/", $entry, $matches) > 0 ) {  
								$sample = $matches[1];
                				echo "<option value=$sample>$sample</option>";
                        	}
                    	} 
					}
                  ?>
             </datalist>
		  </h1>
		  <h1 >Gene 2:
		      <label for="genome"></label>
		      <input name="genome" id="genome" list="gene2">
              <datalist id="gene2">
		        <option value="hg18">hg18</option>
		        <option value="hg19">hg19</option>
	        </datalist>
	        <p>Type of fusions 
		      <label for="filterType"></label>
		      <select name="filterType" id="filterType">
		        <option value="all">all</option>
		        <option value="inter">inter-chromosomal</option>
		        <option value="intra">intra-chromosomal</option>
		        <option value="intra_inter">inter- & intra-chromosomal</option>
		        <option value="read-through">read-through</option>
              </select>
	        </p>
		    <br />
            <input type="hidden" value="normal" name="debug"/>
		    <input type="submit" class="button"/>
	      </h1>
      </form>

--My goal is to populate a text form so an autocmplete function will be available.

--I have accomplished this, but i'm afraid it will take the code a while to go through the 25,000+ files i have in my directory and populate the text form.

--My goal is to find a quicker way for the files to be read directly.

--I have some great suggestions above and i will try them. 

--Do you guys or anyone else have other advice or a php template to achieve my goal?

||!Thanks a lot for reading!!||

--

Edited July 24, 2013 by dev1902

[PaulRyan]

Use a database to store all of the file names and file paths, then do a search on the database instead of searching the directory each time.

Edited July 24, 2013 by PaulRyan

[dev1902]

Use a database to store all of the file names and file paths, then do a search on the database instead of searching the directory each time.

I am not exactly sure how to do that. I have .gfr (gene fusion report) files. I don't really know how to go about doing that using a database to store them.  I am using the XAMPP package with a local server and mysql database but i'm not sure how to go about uploading all 25,000+ files. Can you help me with that?

[AbraCadaver]

I am not exactly sure how to do that. I have .gfr (gene fusion report) files. I don't really know how to go about doing that using a database to store them.  I am using the XAMPP package with a local server and mysql database but i'm not sure how to go about uploading all 25,000+ files. Can you help me with that?

The problem comes in when you add files to that dir.  If you never add files then that is OK.  If you do add/delete files then you would have to update the database every time you add/delete.  If that is a possibilty then this may make sense.

 

BTW, the foreach() glob() that I posted, only took 2.3 seconds for 26,000 files on my laptop with SSD.  Not sure what times you will get.

[Psycho]

Never user regular expressions when you can accomplish the same thing with a simple string function. In this case you could use strrchr() to return the end of the file name after the last period to check the extension. Or, you could use pathinfo() to get the extension as well.

 

But, I agree with PaulRyan completely. You need to create a script that will process the files and extract the necessary information you need and populate it into a database. Trying to read thousands of files on a page load will kill the server.

[QuickOldCar]

If you can now read this data in a file through php, you can make each a variable and make it an array in a loop, then insert them into mysql using columns and fields you come up with.

 

I'm sure you can glob the directory and do each file like I said above, is hard to determine without seeing an example of the data in each file, and how you can seperate the data.

 

Usually people would have a cvs or text file and use mysql's Load Data Infile

 

There are some import tools out there for popular file types, I doubt will find a .gfr one.

 

So with that said, I believe you will have to either do replaces on each of your files and change yourself to a more popular format, or write a script that does this for you. Either way you will have to find the unique data in each of your files so can be inserted into mysql how you would like it.

[AbraCadaver]

I wouldn't put the files in the database, just the names.

[QuickOldCar]

Yes, just doing the names be much simpler, I agree with AbraCadaver's suggestion.

[dev1902]

Thanks a lot guys.

- I will look more into modifying my code for it to read the files. 

- I will look into the database upload also.

- What i am working on is a web database to store information on the human genome.

- The codes i run takes the information from the files in my directory and displays the infromation in categories and tables on two other pages.