Reading from a txt file and searching or entering in to database

dmccabe · December 11, 2008

I have been asking a few questions about separate parts of the same script, but am rapidly getting confused, so figured I would start again and ask the question in full, so please forgive me for the long post.

Ok so what I am doing is allowing a user to upload a logfile that will contain this sort of info:

R1 - HKLM\Software\Microsoft\Internet Explorer\Main,Default_Page_URL = http://go.microsoft.com/fwlink/?LinkId=69157

R1 - HKLM\Software\Microsoft\Internet Explorer\Main,Default_Search_URL = http://go.microsoft.com/fwlink/?LinkId=54896

R1 - HKLM\Software\Microsoft\Internet Explorer\Main,Search Page = http://go.microsoft.com/fwlink/?LinkId=54896

R0 - HKLM\Software\Microsoft\Internet Explorer\Main,Start Page = http://go.microsoft.com/fwlink/?LinkId=69157

R1 - HKCU\Software\Microsoft\Internet Connection Wizard,ShellNext = http://windowsupdate.microsoft.com/

O2 - BHO: Trend Micro Toolbar BHO - {43C6D902-A1C5-45c9-91F6-FD9E90337E18} - C:\Program Files\Trend Micro\TrendSecure\TISProToolbar\TSToolbar.dll

O2 - BHO: Google Toolbar Helper - {AA58ED58-01DD-4d91-8333-CF10577473F7} - C:\Program Files\Google\Google Toolbar\GoogleToolbar.dll

O2 - BHO: Google Toolbar Notifier BHO - {AF69DE43-7D58-4638-B6FA-CE66B5AD205D} - C:\Program Files\Google\GoogleToolbarNotifier\4.1.509.5470\swg.dll

O3 - Toolbar: Trend Micro Toolbar - {CCAC5586-44D7-4c43-B64A-F042461A97D2} - C:\Program Files\Trend Micro\TrendSecure\TISProToolbar\TSToolbar.dll

O3 - Toolbar: &Google Toolbar - {2318C2B1-4965-11d4-9B18-009027A5CD4F} - C:\Program Files\Google\Google Toolbar\GoogleToolbar.dll

O4 - HKLM\..\Run: [RTHDCPL] RTHDCPL.EXE

O4 - HKLM\..\Run: [skyTel] SkyTel.EXE

O4 - HKLM\..\Run: [Alcmtr] ALCMTR.EXE

O4 - HKLM\..\Run: [NvCplDaemon] RUNDLL32.EXE C:\WINDOWS\system32\NvCpl.dll,NvStartup

O4 - HKLM\..\Run: [nwiz] nwiz.exe /install

O4 - HKLM\..\Run: [NvMediaCenter] RUNDLL32.EXE C:\WINDOWS\system32\NvMcTray.dll,NvTaskbarInit

O4 - HKLM\..\Run: [TrueImageMonitor.exe] C:\Program Files\Acronis\TrueImage\TrueImageMonitor.exe

O4 - HKLM\..\Run: [Acronis Scheduler2 Service] "C:\Program Files\Common Files\Acronis\Schedule2\schedhlp.exe"

O4 - HKLM\..\Run: [QuickTime Task] "C:\Program Files\QuickTime\qttask.exe" -atboottime

O4 - HKLM\..\Run: [ufSeAgnt.exe] "C:\Program Files\Trend Micro\Internet Security\UfSeAgnt.exe"

O4 - HKCU\..\Run: [ctfmon.exe] C:\WINDOWS\system32\ctfmon.exe

O4 - HKCU\..\Run: [swg] C:\Program Files\Google\GoogleToolbarNotifier\GoogleToolbarNotifier.exe

O4 - HKCU\..\Run: [OE] C:\Program Files\Trend Micro\Internet Security\TMAS_OE\TMAS_OEMon.exe

O4 - HKUS\S-1-5-19\..\Run: [startUp This] "C:\Program Files\Laplink\PCmover\LaunchSt.exe" (User 'LOCAL SERVICE')

O4 - HKUS\S-1-5-19\..\Run: [OE] C:\Program Files\Trend Micro\Internet Security\TMAS_OE\TMAS_OEMon.exe (User 'LOCAL SERVICE')

O4 - HKUS\S-1-5-20\..\Run: [startUp This] "C:\Program Files\Laplink\PCmover\LaunchSt.exe" (User 'NETWORK SERVICE')

O4 - HKUS\S-1-5-21-1220945662-448539723-682003330-1004\..\Run: [startUp This] "C:\Program Files\Laplink\PCmover\LaunchSt.exe" (User 'Gail')

O4 - HKUS\S-1-5-21-1220945662-448539723-682003330-1005\..\Run: [startUp This] "C:\Program Files\Laplink\PCmover\LaunchSt.exe" (User 'Laura')

O4 - HKUS\S-1-5-18\..\Run: [startUp This] "C:\Program Files\Laplink\PCmover\LaunchSt.exe" (User 'SYSTEM')

O4 - HKUS\.DEFAULT\..\Run: [startUp This] "C:\Program Files\Laplink\PCmover\LaunchSt.exe" (User 'Default user')

O4 - Global Startup: ExifLauncher2.lnk = C:\Program Files\FinePixViewer\QuickDCF2.exe

O8 - Extra context menu item: E&xport to Microsoft Excel - res://C:\PROGRA~1\MICROS~3\OFFICE11\EXCEL.EXE/3000

O9 - Extra button: (no name) - {e2e2dd38-d088-4134-82b7-f2ba38496583} - C:\WINDOWS\Network Diagnostic\xpnetdiag.exe

O9 - Extra 'Tools' menuitem: @xpsp3res.dll,-20001 - {e2e2dd38-d088-4134-82b7-f2ba38496583} - C:\WINDOWS\Network Diagnostic\xpnetdiag.exe

O9 - Extra button: Messenger - {FB5F1910-F110-11d2-BB9E-00C04F795683} - C:\Program Files\Messenger\msmsgs.exe

O9 - Extra 'Tools' menuitem: Windows Messenger - {FB5F1910-F110-11d2-BB9E-00C04F795683} - C:\Program Files\Messenger\msmsgs.exe

O16 - DPF: {0CCA191D-13A6-4E29-B746-314DEE697D83} (Facebook Photo Uploader 5 Control) - http://upload.facebook.com/controls/2008.10.10_v5.5.8/FacebookPhotoUploader5.cab

O16 - DPF: {0E5F0222-96B9-11D3-8997-00104BD12D94} (PCPitstop Utility) - http://www.pcpitstop.com/betapit/PCPitStop.CAB

O16 - DPF: {1E54D648-B804-468d-BC78-4AFFED8E262E} (System Requirements Lab) - http://www.nvidia.com/content/DriverDownload/srl/3.0.0.0/srl_bin/sysreqlab3.cab

O16 - DPF: {215B8138-A3CF-44C5-803F-8226143CFC0A} (Trend Micro ActiveX Scan Agent 6.6) - http://housecall65.trendmicro.com/housecall/applet/html/native/x86/win32/activex/hcImpl.cab

O16 - DPF: {2D8ED06D-3C30-438B-96AE-4D110FDC1FB8} (ActiveScan 2.0 Installer Class) - http://acs.pandasoftware.com/activescan/cabs/as2stubie.cab

O16 - DPF: {6414512B-B978-451D-A0D8-FCFDF33E833C} (WUWebControl Class) - http://www.update.microsoft.com/windowsupdate/v6/V5Controls/en/x86/client/wuweb_site.cab?1221145738406

O16 - DPF: {6E32070A-766D-4EE6-879C-DC1FA91D2FC3} (MUWebControl Class) - http://www.update.microsoft.com/microsoftupdate/v6/V5Controls/en/x86/client/muweb_site.cab?1218347998289

O16 - DPF: {74DBCB52-F298-4110-951D-AD2FF67BC8AB} (NVIDIA Smart Scan) - http://www.nvidia.com/content/DriverDownload/nforce/NvidiaSmartScan.cab

O16 - DPF: {D27CDB6E-AE6D-11CF-96B8-444553540000} (Shockwave Flash Object) - http://fpdownload2.macromedia.com/get/shockwave/cabs/flash/swflash.cab

O18 - Protocol: tmtb - {04EAF3FB-4BAC-4B5A-A37D-A1CF210A5A42} - C:\Program Files\Trend Micro\TrendSecure\TISProToolbar\TSToolbar.dll

O23 - Service: Lavasoft Ad-Aware Service (aawservice) - Lavasoft - C:\Program Files\Lavasoft\Ad-Aware\aawservice.exe

O23 - Service: Acronis Scheduler2 Service (AcrSch2Svc) - Acronis - C:\Program Files\Common Files\Acronis\Schedule2\schedul2.exe

O23 - Service: Google Updater Service (gusvc) - Google - C:\Program Files\Google\Common\Google Updater\GoogleUpdaterService.exe

O23 - Service: NVIDIA Display Driver Service (NVSvc) - NVIDIA Corporation - C:\WINDOWS\system32\nvsvc32.exe

O23 - Service: Cyberlink RichVideo Service(CRVS) (RichVideo) - Unknown owner - C:\Program Files\CyberLink\Shared Files\RichVideo.exe

O23 - Service: Security Activity Dashboard Service - Trend Micro Inc. - C:\Program Files\Trend Micro\TrendSecure\SecurityActivityDashboard\tmarsvc.exe

O23 - Service: Trend Micro Central Control Component (SfCtlCom) - Trend Micro Inc. - C:\Program Files\Trend Micro\Internet Security\SfCtlCom.exe

O23 - Service: Trend Micro Unauthorized Change Prevention Service (TMBMServer) - Trend Micro Inc. - C:\Program Files\Trend Micro\BM\TMBMSRV.exe

O23 - Service: Trend Micro Personal Firewall (TmPfw) - Trend Micro Inc. - C:\Program Files\Trend Micro\Internet Security\TmPfw.exe

O23 - Service: Trend Micro Proxy Service (TmProxy) - Trend Micro Inc. - C:\Program Files\Trend Micro\Internet Security\TmProxy.exe

The files are all structured identically and will all contain similar information.

I have then set up my db, with a separate table for each prefix:

tbl_R0 (ID, Path, Count)

tbl_R1 (ID, Path, Count)

tbl_R2 (ID, Path, Count)

tbl_O1 (ID, Path, Count)

etc

Now what I want to happen when they upload the file is for it to take each line at a time, and split it in to 2 parts:

1 - The prefix, eg: R1, R0, O1, O2, etc etc

2 - The path, eg: HKLM\Software\Microsoft\Internet Explorer\Main,Default_Page_URL = http://go.microsoft.com/fwlink/?LinkId=69157

It then takes the line it is analysing, and searches the relevant table tbl_prefix to see if that path exists in that table.

If it is not there it should add it to the table and increase the count by 1

If it is there, then it should increase the count by 1

Now I have gone some way in getting this right, but I doubt very much I have gone about it the right way and rather than analyse my code below, I would like to hear suggestions on doing this the right way.

Please bear in mind when making suggestions, I intend this to process a lot of logs, so it needs to efficient.

At the moment I have it in 2 files, one is just the upload form and the other is the upload.php script below:

<?php
include("includes/dbconnect.php");  //connect to the Database

$target_path = "templogs/"; //set path for uploads to be stored in
$target_path = $target_path . basename( $_FILES['uploadedfile']['name']); // set full path and filename

echo "Techmonkeys HiJackThis Logfile Analysis V0.1b";

if(move_uploaded_file($_FILES['uploadedfile']['tmp_name'], $target_path)) {  							//Check if file uploads correctly
$randfn = createRandomfilename(); 																	// Generate Random file name
if (rename('templogs/'.  basename( $_FILES['uploadedfile']['name']), 'logs/'. $randfn .'.log')) {  	//rename the file to random file name & move it
	$randfn = "logs/". $randfn .".log";																// Set the path including random file name
	$handle = fopen($randfn, 'r');																	// Open the file for reading
	echo "<table>";
	$a = 1;																							// Set i to 1 to perform different actions  on different lines.
	while (!feof($handle))																			// Begin whille loop until end of file
	{
		$data = fgets($handle, 512);																// set the contents of $data to the current line in the file
		//echo "<tr><td>$a </td><td> row</td></tr>";
		if ($a < 2) {																				// Check to see if it is the first line	
			$ver = "2.0.2";																			// **** need to replace with version from db *****
			//echo "<tr><td><strong>Version<strong></td><td align='left'>";
			if(strstr($data,$ver)) {																// find version number in current line
				//echo "v2.0.2 - Correct";
			} else {
				//echo "Incorrect, please download the latest version from <A href='http://www.trendsecure.com/portal/en-US/tools/security_tools/hijackthis'>Trend Micro</a>";
			}
			//echo "</td></tr>";
		}
		if ((left($data,1) == "O" || left($data,1) == "R" || left($data,1) == "N" || left($data,1) == "F")  && (left($data,2) != "Ru")) {													// Check to see that the first character is not C and that it isnt a blank line
			$section = explode(" - ",$data);
			$tbl_no = "tbl_". $section[0];
			preg_match("/-\s(.*)/",$data,$path);
			$srchquery = "SELECT * FROM `". $tbl_no . "` WHERE `path` = '". $path[1] ."'";
			$result = mysql_query($srchquery);
			echo "tbl_no". $tbl_no . " - ". addslashes($path[1]) ."<br />";
			if (!$result) {
				//echo "No results found<br />";
				$insquery = "INSERT into `". $tbl_no ."` VALUES ('','". addslashes($path[1]) ."','1')";
				$result = mysql_query($insquery) or die(mysql_error());
			}
			else {
				$row = mysql_fetch_array($result);
				$count = $row['count'];
				$count++;
				//echo "<tr></td>". $tbl_no ." - ". $path[1] ."</td><td>Seen <strong>". $count ." times.</strong></td></tr>";
			}

		}
		$a++;
	}
	echo "</table>";
	fclose($handle); 																				// Close the file
} else {
	echo "<strong><font color='#FF0000'>Error:</font></strong> File was not renamed";
}
} else{
    echo "There was an error uploading the file, please try again!";
}

function createRandomfilename() {     																	// Function to create a random file name
   $chars = "abcdefghijkmnopqrstuvwxyz023456789";
   srand((double)microtime()*1000000);
   $i = 0;
   $pass = '' ;
   while ($i <= 7) {
        $num = rand() % 33;
        $tmp = substr($chars, $num, 1);
        $pass = $pass . $tmp;
        $i++;
   }
   return $pass;
}


function left($str, $length) {																			// Function for trimming from left
return substr($str, 0, $length);
}


mysql_close();																							// Close DB connection


?>

Thanks for you help in advance

dmccabe · December 11, 2008

Sorry I should have said, the above code doesn't do what it should at the moment, except with the lines that begin with a O4 prefix, for some reason it does input those in to the database in the right table. However if I run it again with the same file, it just enters them again, it doesnt find them and increase the count.

Maq · December 11, 2008

For some reason when I copied your code it put a massive amount of linebreaks in my IDE (it's over 1,000 lines now ;/). Could you attach a .txt file or something?

dmccabe · December 11, 2008

No probs here you go!

[attachment deleted by admin]

dmccabe · December 12, 2008

Anyone?

dmccabe · December 12, 2008

Sorry for the bumps guys, but am really struggling with this.

If anyone can give me any clues I would appreciate it

Sign In

Reading from a txt file and searching or entering in to database

Recommended Posts

dmccabe

Link to comment

Share on other sites

dmccabe

Link to comment

Share on other sites

Maq

Link to comment

Share on other sites

dmccabe

Link to comment

Share on other sites

dmccabe

Link to comment

Share on other sites

dmccabe

Link to comment

Share on other sites

Join the conversation

Browse

Activity

Important Information