Jump to content

Help with directory loop


cg73
Go to solution Solved by Psycho,

Recommended Posts

Hello!  Please forgive me as I am struggling to learn php now for a while. 

I have been trying to use php to insert my nmap xml result files into a mysql database, and have been able to get a single file to input correctly using existing code from the internet, however no matter what I have tried I can't get a loop to work properly when trying to modify to read several xml files in a particular directory.  

I have experimented with GLOB - 

$file = glob("directory/*"); 

foreach ($file as $line) {

My screen stays blank and no db insert.

I have tried using "scandir", and I still get a blank screen and no db insert.

$directory = 'directory';
$files = scandir($directory);
foreach ($files as $file) {

Is there another method I can use for reading the values contained within my xml files, or can anyone suggest where my code is not right please?  

I have pasted the entire code below and would be grateful for any assistance provided:

<?php

$file = file('myfile.xml');      *** I have tried using various array options, and the filenames do load into the array, tested with printr_($file[1]) etc.  But the foreach code appears to ignore all files, even the 1st one. ***

$servername = "localhost";
$username = "dbuser";
$password = "dbpass";
$db = "dbdb";

$conn = new mysqli($servername, $username, $password, $db);

if ($conn->connect_error){
    die("Connection failed: ". $conn->connect_error);
}

$ip;
$hostname;
$port;
$portArray = array();
$portList;
$timestamp;

foreach($file as $line){

    //Get IP Address
    if (strpos($line, 'addrtype="ipv4"') == TRUE){
    preg_match('/addr=".* addrtype/',$line,$results);
    $ip = implode(" ",$results);
    $ip = ltrim($ip, 'addr="');
    $ip = rtrim($ip, '" addrtype');
    print "<br><strong><u>Device</u></strong><br>";
    print "IP Address:  $ip<br>";
    }

    //Get Hostname
    if (strpos($line, 'type="PTR"') == TRUE){
    preg_match('/name=".*" type/',$line,$results);
    $hostname = implode(" ",$results);
    $hostname = ltrim($hostname,'name="');
    $hostname = rtrim($hostname, ' type');
    $hostname = rtrim($hostname, '"');
    print "Hostname:  $hostname<br>";
    }

    //Get Ports
    if (strpos($line, 'portid="') == TRUE){
    preg_match('/portid=".*><state/',$line,$results);
    $port = implode(" ",$results);
    $port = ltrim($port,'portid="');
    $port = rtrim($port, '"><state');
    print "Port: $port<br>";
    array_push($portArray, $port);
    }

    //Add Values to Database
    if (strpos($line, '/host>') == TRUE){
    $timestamp = time();
    $mytime = new \DateTimeImmutable('@'.$timestamp);
    $portList = implode(", ",$portArray);
    $sql = "insert into _____ (ip,hostname,ports,timestamp) values ('$ip','$hostname','$portList','$timestamp')";

    if ($conn->query($sql) === TRUE) {
        echo "Data Added: $ip  - $hostname - $portList - $timestamp <br>";
    } else {
        echo "Error: ".$sql."<br>".$conn->error;
    }
    $ip = " ";
    $hostname = " ";
    unset($portArray);
    $portArray = array();
    $portList = " ";
    }

}

$conn->close();

?>

 

 

Thank you!

 

 

Edited by Psycho
Put code into code tags
Link to comment
Share on other sites

  • Solution

A couple things. First please put code into code tags using the forum editor. I have fixed your initial post.

Second, you say that you get a "blank screen", but you have provided no content of what debugging you have performed. You state you are looking for another way to iterate over the files - but you have already confirmed that the files do load and you can iterate over them - you just aren't getting results. So, rather than finding a different way to get the files (which you have already accomplished), you need to figure out why it is not working.

I suspect the issue may be your variable names are causing confusion and you aren't passing what you think you are to the processing code. In example you post with a single file you have this:

$file = file('myfile.xml'); 

In that case $file is an array of the contents of the file. Then you state that you tried thing such as

$file = glob("directory/*");
foreach ($file as $line) {

or

$directory = 'directory';
$files = scandir($directory);
foreach ($files as $file) {

In the first case $line is a $file (not a line) and in the second case $file (in the foreach loop) is a reference to the file NOT an array of the contents of the file. I'm assuming you are not calling file() on the file reference within your loop. If you had called that first variable $linesAry, which is more accurate to what it is, I think you would have seen this yourself.

This should work. Note I don't have a an environment to test on at the moment. So there could be some minor typos\errors.

<?php

//Make the DB connection
$servername = "localhost";
$username = "dbuser";
$password = "dbpass";
$db = "dbdb";
$conn = new mysqli($servername, $username, $password, $db);

if ($conn->connect_error){
    die("Connection failed: ". $conn->connect_error);
}

//Get the contents of the directory
$directoryStr = "directory/*";
echo "Processing directory: {$directoryStr}<br>\n";
$filesAry = glob($directoryStr); 
echo "Found: " . count($filesAry) . " files<br><br>\n";

//Iterate over each file
foreach ($filesAry as $fileStr)
{
	//Reset the variables for the data being searched
	$ip = '';
	$hostname = '';
	$port = '';
	$portArray = array();
	$portList = '';
	$timestamp = '';
	
	//Get content of the file into an array
	echo "Processing file: {$fileStr}<br>\n";
	$linesAry = file($fileStr);

	//Iterate over each line of text in the file
	foreach($linesAry as $lineStr)
	{

		//Get IP Address
		if (strpos($lineStr, 'addrtype="ipv4"') == TRUE)
		{
			preg_match('/addr=".* addrtype/', $lineStr, $results);
			$ip = implode(" ",$results);
			$ip = ltrim($ip, 'addr="');
			$ip = rtrim($ip, '" addrtype');
			echo "<br><strong><u>Device</u></strong><br>";
			echo "IP Address:  $ip<br>";
		}

		//Get Hostname
		if (strpos($lineStr, 'type="PTR"') == TRUE)
		{
			preg_match('/name=".*" type/',$lineStr,$results);
			$hostname = implode(" ",$results);
			$hostname = ltrim($hostname,'name="');
			$hostname = rtrim($hostname, ' type');
			$hostname = rtrim($hostname, '"');
			echo "Hostname:  $hostname<br>";
		}

		//Get Ports
		if (strpos($lineStr, 'portid="') == TRUE)
		{
			preg_match('/portid=".*><state/',$lineStr,$results);
			$port = implode(" ",$results);
			$port = ltrim($port,'portid="');
			$port = rtrim($port, '"><state');
			echo "Port: $port<br>";
			array_push($portArray, $port);
		}

		//Add Values to Database
		if (strpos($lineStr, '/host>') == TRUE)
		{
			$timestamp = time();
			$mytime = new \DateTimeImmutable('@'.$timestamp);
			$portList = implode(", ",$portArray);
			$sql = "insert into _____ (ip,hostname,ports,timestamp) values ('$ip', '$hostname', '$portList', '$timestamp')";

			if ($conn->query($sql) === TRUE) {
				echo "Data Added: $ip  - $hostname - $portList - $timestamp <br>";
			} else {
				echo "Error: ".$sql."<br>".$conn->error;
			}
		}
	}
}

//Close the DB connection
$conn->close();

?>

 

  • Like 1
Link to comment
Share on other sites

Thanks Psycho!

I have read through your suggested modifications and yes, I understand now using $line vs $lineAry.  Also thank you for adding the comments expanding with the additions such as echoing the Found: etc.  It helps me understand a lot clearer.

Good news, I have made the changes and tested the new code.  It works very well, however I must troubleshoot the portArray section, as I am now getting duplicate port numbers and hostnames inserted from the xml files.  Thought it may have been multiple nmap scan xml files (different days with IP's changing dynamically which would account for multiple hostnames showing up), but when I try the code with a single xml nmap output I still have multiple ip's showing the same hostname, and the port lists are doubled up.  Strangely, the order is not repetitive though.

 

Data Added: 192.168.x.x - xx:xx:xx:xx:xx:xx - LCFC(HeFei) Electronics Technology - xxxxxxxxxx.local - 22, 80, 443, 80, 80, 80, 22, 80, 443, 22, 80, 443, 7, 9, 13, 17, 19, 135, 139, 5800, 5900, 27000, 21, 22, 80, 111, 139, 443, 445, 548, 554, 2049, 3261, 5000, 5001, 9900, 80, 443, 5060, 5061, 139, 445, 3389, 80, 443, 554, 1935, 6001, 8000, 9000, 80, 443, 554, 1935, 6001, 8000, 9000, 139, 445, 3389, 254, 616, 631, 726, 777, 1044, 1148, 1556, 1998, 2179, 2998, 3546, 4899, 4900, 5061, 5087, 6106, 7103, 15004, 49152, 56738, 62078, 22, 80, 139, 445, 3389, 135, 139, 445, 3306, 3389, 80, 443, 135, 139, 445, 3389, 139, 445, 80, 135, 139, 443, 445, 1311, 3306, 3389, 8042, 135, 139, 445, 80, 111, 139, 443, 445, 554, 2049, 3260, 3261, 4045, 5000, 5001, 5357, 5510, 9900, 139, 445, 3389, 135, 139, 445, 3389, 139, 445, 3389, 8080, 22, 80, 443, 5900, 139, 445, 3389, 80, 443, 554, 1935, 6001, 8000, 9000, 53, 80, 135, 139, 445, 1433, 1801, 2103, 2105, 2107, 3389, 8002, 9102, 135, 139, 443, 445, 2179, 3389, 80, 443, 139, 445, 3389, 80, 111, 139, 443, 445, 554, 2049, 3260, 3261, 4045, 5000, 5001, 5357, 5510, 9900, 53, 80, 135, 139, 445, 1433, 1801, 2103, 2105, 2107, 3389, 8002, 9102, 139, 445, 3389, 21, 80, 111, 139, 445, 2049, 5357, 8001, 8002, 49154, 80, 443, 3389, 139, 445, 3389, 80, 443, 139, 445, 3389, 135, 139, 445, 2000, 2100, 2638, 139, 445, 3389, 139, 445, 135, 139, 445, 5357, 25, 80, 135, 139, 445, 1311, 1433, 2525, 3030, 3389, 25734, 25735, 139, 445, 3389 - 1703125643

 

Thank you, I appreciate your time, effort and quick response.  I have learned from your suggestions.

 

 

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.