Help with directory loop

[cg73]

Hello!  Please forgive me as I am struggling to learn php now for a while. 

I have been trying to use php to insert my nmap xml result files into a mysql database, and have been able to get a single file to input correctly using existing code from the internet, however no matter what I have tried I can't get a loop to work properly when trying to modify to read several xml files in a particular directory.  

I have experimented with GLOB - 

$file = glob("directory/*"); 

foreach ($file as $line) {

My screen stays blank and no db insert.

I have tried using "scandir", and I still get a blank screen and no db insert.

$directory = 'directory';
$files = scandir($directory);
foreach ($files as $file) {

Is there another method I can use for reading the values contained within my xml files, or can anyone suggest where my code is not right please?  

I have pasted the entire code below and would be grateful for any assistance provided:

<?php

$file = file('myfile.xml');      *** I have tried using various array options, and the filenames do load into the array, tested with printr_($file[1]) etc.  But the foreach code appears to ignore all files, even the 1st one. ***

$servername = "localhost";
$username = "dbuser";
$password = "dbpass";
$db = "dbdb";

$conn = new mysqli($servername, $username, $password, $db);

if ($conn->connect_error){
    die("Connection failed: ". $conn->connect_error);
}

$ip;
$hostname;
$port;
$portArray = array();
$portList;
$timestamp;

foreach($file as $line){

    //Get IP Address
    if (strpos($line, 'addrtype="ipv4"') == TRUE){
    preg_match('/addr=".* addrtype/',$line,$results);
    $ip = implode(" ",$results);
    $ip = ltrim($ip, 'addr="');
    $ip = rtrim($ip, '" addrtype');
    print "<br><strong><u>Device</u></strong><br>";
    print "IP Address:  $ip<br>";
    }

    //Get Hostname
    if (strpos($line, 'type="PTR"') == TRUE){
    preg_match('/name=".*" type/',$line,$results);
    $hostname = implode(" ",$results);
    $hostname = ltrim($hostname,'name="');
    $hostname = rtrim($hostname, ' type');
    $hostname = rtrim($hostname, '"');
    print "Hostname:  $hostname<br>";
    }

    //Get Ports
    if (strpos($line, 'portid="') == TRUE){
    preg_match('/portid=".*><state/',$line,$results);
    $port = implode(" ",$results);
    $port = ltrim($port,'portid="');
    $port = rtrim($port, '"><state');
    print "Port: $port<br>";
    array_push($portArray, $port);
    }

    //Add Values to Database
    if (strpos($line, '/host>') == TRUE){
    $timestamp = time();
    $mytime = new \DateTimeImmutable('@'.$timestamp);
    $portList = implode(", ",$portArray);
    $sql = "insert into _____ (ip,hostname,ports,timestamp) values ('$ip','$hostname','$portList','$timestamp')";

    if ($conn->query($sql) === TRUE) {
        echo "Data Added: $ip  - $hostname - $portList - $timestamp <br>";
    } else {
        echo "Error: ".$sql."<br>".$conn->error;
    }
    $ip = " ";
    $hostname = " ";
    unset($portArray);
    $portArray = array();
    $portList = " ";
    }

}

$conn->close();

?>

 

 

Thank you!

 

 

Edited December 20, 2023 by Psycho
Put code into code tags

[Barand]

Take look at simpleXML.

[Psycho]

A couple things. First please put code into code tags using the forum editor. I have fixed your initial post.

Second, you say that you get a "blank screen", but you have provided no content of what debugging you have performed. You state you are looking for another way to iterate over the files - but you have already confirmed that the files do load and you can iterate over them - you just aren't getting results. So, rather than finding a different way to get the files (which you have already accomplished), you need to figure out why it is not working.

I suspect the issue may be your variable names are causing confusion and you aren't passing what you think you are to the processing code. In example you post with a single file you have this:

$file = file('myfile.xml'); 

In that case $file is an array of the contents of the file. Then you state that you tried thing such as

$file = glob("directory/*");
foreach ($file as $line) {

or

$directory = 'directory';
$files = scandir($directory);
foreach ($files as $file) {

In the first case $line is a $file (not a line) and in the second case $file (in the foreach loop) is a reference to the file NOT an array of the contents of the file. I'm assuming you are not calling file() on the file reference within your loop. If you had called that first variable $linesAry, which is more accurate to what it is, I think you would have seen this yourself.

This should work. Note I don't have a an environment to test on at the moment. So there could be some minor typos\errors.

<?php

//Make the DB connection
$servername = "localhost";
$username = "dbuser";
$password = "dbpass";
$db = "dbdb";
$conn = new mysqli($servername, $username, $password, $db);

if ($conn->connect_error){
    die("Connection failed: ". $conn->connect_error);
}

//Get the contents of the directory
$directoryStr = "directory/*";
echo "Processing directory: {$directoryStr}<br>\n";
$filesAry = glob($directoryStr); 
echo "Found: " . count($filesAry) . " files<br><br>\n";

//Iterate over each file
foreach ($filesAry as $fileStr)
{
	//Reset the variables for the data being searched
	$ip = '';
	$hostname = '';
	$port = '';
	$portArray = array();
	$portList = '';
	$timestamp = '';
	
	//Get content of the file into an array
	echo "Processing file: {$fileStr}<br>\n";
	$linesAry = file($fileStr);

	//Iterate over each line of text in the file
	foreach($linesAry as $lineStr)
	{

		//Get IP Address
		if (strpos($lineStr, 'addrtype="ipv4"') == TRUE)
		{
			preg_match('/addr=".* addrtype/', $lineStr, $results);
			$ip = implode(" ",$results);
			$ip = ltrim($ip, 'addr="');
			$ip = rtrim($ip, '" addrtype');
			echo "<br><strong><u>Device</u></strong><br>";
			echo "IP Address:  $ip<br>";
		}

		//Get Hostname
		if (strpos($lineStr, 'type="PTR"') == TRUE)
		{
			preg_match('/name=".*" type/',$lineStr,$results);
			$hostname = implode(" ",$results);
			$hostname = ltrim($hostname,'name="');
			$hostname = rtrim($hostname, ' type');
			$hostname = rtrim($hostname, '"');
			echo "Hostname:  $hostname<br>";
		}

		//Get Ports
		if (strpos($lineStr, 'portid="') == TRUE)
		{
			preg_match('/portid=".*><state/',$lineStr,$results);
			$port = implode(" ",$results);
			$port = ltrim($port,'portid="');
			$port = rtrim($port, '"><state');
			echo "Port: $port<br>";
			array_push($portArray, $port);
		}

		//Add Values to Database
		if (strpos($lineStr, '/host>') == TRUE)
		{
			$timestamp = time();
			$mytime = new \DateTimeImmutable('@'.$timestamp);
			$portList = implode(", ",$portArray);
			$sql = "insert into _____ (ip,hostname,ports,timestamp) values ('$ip', '$hostname', '$portList', '$timestamp')";

			if ($conn->query($sql) === TRUE) {
				echo "Data Added: $ip  - $hostname - $portList - $timestamp <br>";
			} else {
				echo "Error: ".$sql."<br>".$conn->error;
			}
		}
	}
}

//Close the DB connection
$conn->close();

?>

 

[cg73]

Thanks Psycho!

I have read through your suggested modifications and yes, I understand now using $line vs $lineAry.  Also thank you for adding the comments expanding with the additions such as echoing the Found: etc.  It helps me understand a lot clearer.

Good news, I have made the changes and tested the new code.  It works very well, however I must troubleshoot the portArray section, as I am now getting duplicate port numbers and hostnames inserted from the xml files.  Thought it may have been multiple nmap scan xml files (different days with IP's changing dynamically which would account for multiple hostnames showing up), but when I try the code with a single xml nmap output I still have multiple ip's showing the same hostname, and the port lists are doubled up.  Strangely, the order is not repetitive though.

 

Data Added: 192.168.x.x - xx:xx:xx:xx:xx:xx - LCFC(HeFei) Electronics Technology - xxxxxxxxxx.local - 22, 80, 443, 80, 80, 80, 22, 80, 443, 22, 80, 443, 7, 9, 13, 17, 19, 135, 139, 5800, 5900, 27000, 21, 22, 80, 111, 139, 443, 445, 548, 554, 2049, 3261, 5000, 5001, 9900, 80, 443, 5060, 5061, 139, 445, 3389, 80, 443, 554, 1935, 6001, 8000, 9000, 80, 443, 554, 1935, 6001, 8000, 9000, 139, 445, 3389, 254, 616, 631, 726, 777, 1044, 1148, 1556, 1998, 2179, 2998, 3546, 4899, 4900, 5061, 5087, 6106, 7103, 15004, 49152, 56738, 62078, 22, 80, 139, 445, 3389, 135, 139, 445, 3306, 3389, 80, 443, 135, 139, 445, 3389, 139, 445, 80, 135, 139, 443, 445, 1311, 3306, 3389, 8042, 135, 139, 445, 80, 111, 139, 443, 445, 554, 2049, 3260, 3261, 4045, 5000, 5001, 5357, 5510, 9900, 139, 445, 3389, 135, 139, 445, 3389, 139, 445, 3389, 8080, 22, 80, 443, 5900, 139, 445, 3389, 80, 443, 554, 1935, 6001, 8000, 9000, 53, 80, 135, 139, 445, 1433, 1801, 2103, 2105, 2107, 3389, 8002, 9102, 135, 139, 443, 445, 2179, 3389, 80, 443, 139, 445, 3389, 80, 111, 139, 443, 445, 554, 2049, 3260, 3261, 4045, 5000, 5001, 5357, 5510, 9900, 53, 80, 135, 139, 445, 1433, 1801, 2103, 2105, 2107, 3389, 8002, 9102, 139, 445, 3389, 21, 80, 111, 139, 445, 2049, 5357, 8001, 8002, 49154, 80, 443, 3389, 139, 445, 3389, 80, 443, 139, 445, 3389, 135, 139, 445, 2000, 2100, 2638, 139, 445, 3389, 139, 445, 135, 139, 445, 5357, 25, 80, 135, 139, 445, 1311, 1433, 2525, 3030, 3389, 25734, 25735, 139, 445, 3389 - 1703125643

 

Thank you, I appreciate your time, effort and quick response.  I have learned from your suggestions.

 

 

[cg73]

Further investigation it seems the portArray is continually expanding with prior results appended to the array as it progresses through the xml file. 

Edited December 21, 2023 by cg73

[cg73]

Edit:  I have added a $portArray clear/reset at the end of the Mysql insert and that seems to have worked wonderfully.

Thanks again for all of your assistance!