Read Text Files

kney · June 6, 2011

Hey,

First, I'm going to tell you what I've done already and then I'm going to show you what still needs to be done.

I'm getting certain information out of a text file (tokenID's & runtimes per token)

In this example I take the runtime for each step (here it will be 9953 + 10903 + 65456 + 83128 + 12187 + 28148 = 209775 (ms)) and in total it will be 209775 for that specific tokenID.

Now, it displays the tokenID + total time per token

Example of how the textfile looks like (actually it's a little complicater than this but it's the brushed up version)

metadataIngester: now executing performAction with token: 9188147351
metadataIngester: result of performAction with token:9188147351
metadataIngester: runtime (ms) for token 9188147351: 9953
inferencer: now executing performAction with token:9188147351
inferencer: result of performAction with token:9188147351
inferencer: runtime (ms) for token 9188147351: 10903
transcoder: now executing performAction with token:9188147351
transcoder: result of performAction with token:9188147351
transcoder: runtime (ms) for token 9188147351: 65456
transcoder: now executing performAction with token:9188147351
transcoder: result of performAction with token:9188147351
transcoder: runtime (ms) for token 9188147351: 83128
streamingServerTransfer: now executing performAction with token:9188147351
streamingServerTransfer: result of performAction with token:9188147351
streamingServerTransfer: runtime (ms) for token 9188147351: 12187
middleware: now executing performAction with token:9188147351
middleware: result of performAction with token:9188147351
middleware: runtime (ms) for token 9188147351: 28148

Now, everything works except 1 small thing I forgot and I don't know how to build it in.

I would like to show every fase he goes through (metadataIngester, inferencer etc.) BUT I should only show 1 runtime per fase, the first I encounter (in this example there are more per fase)

This is the code I have:

<?php

function parseTXT($data){
$file = "server.log";
$fh = fopen($file, "r");
$data = fread($fh,filesize($file));
fclose($fh);
$d_array = explode("\n", $data);
foreach ($d_array as $val){
	if (strstr($val, "runtime (ms) for token")){
		$tmp_array = explode(" ", $val);
		$token = str_replace(":", "", $tmp_array[10]);
		$token_array[$token][] = $tmp_array[11];
		//$token_array2[$token][] = $tmp_array[5];
	}

}

foreach ($token_array as $key=>$val2){
	$token_array[$key]['total'] = 0;
	foreach ($val2 as $val3){
		$token_array[$key]['total'] += $val3;
	}
}
return $token_array;

}

echo "<table border=\"1\"><tr><th>Token ID</th><th>Total Time</th></tr>";
foreach (parseTXT("some_file.txt") as $key=>$val){
foreach ($val as $key2=>$val2){
	if (!is_int($key2)){
		echo "<tr><td>$key</td><td>$val2</td></tr>";
	}
}
}
echo "</table>";
?>

I get this output:

Token 9148825717:
MS to Complete each substep
7348
10715
11162
6334
40840
15477
37840
Total time for 9148825717: 129716 (ms)

But I should get this:

Token 9148825717:
MS to Complete each substep
metadataIngester: 7348
inferencer: 10715
transcoder: 11162
streamingServerTransfer: 15477
middleware: 37840
Total time for 9148825717: 82542 (ms)

WebStyles · June 6, 2011

well, if you read the text file into a variable (get_file_contents('filename.txt') is the prefered method.) you can then split it into lines. then if you split each line into words, you just need to grab the first word and use a variable to store that so you can compare if it repeats or not.

something like this:

$file = file_get_contents('filename.txt');
$lines = explode("\n",$file); // assuming \n is the used linebreak
foreach($lines as $line){
$words = explode(" ",$line);
$firstWord = $words[0];
// do some processing and store $firstWord in a variable like: $lastWordUsed.
}

then all you need to do is check if $firstWord is equal or not to $lastWordUsed.

Hope this helps

kney · June 6, 2011

Thanks for the reply.

But I'm having some trouble implementing it into my code.

fugix · June 6, 2011

can you show use the code that you have now.

kney · June 6, 2011

it's in the first post, I just need the code he suggested implemented

WebStyles · June 6, 2011

so you're not even going to try? you want one of us to do it for you? that won't help you learn.

kney · June 6, 2011

well I really do want to learn but I'm not so great with PHP and also got tons of other stuff to do

&

the textfile like this is more like the original.. so a lil bit more complicated

2011-05-02 18:14:13,350 INFO  [com.siemens.vcms.actor.common.JbpmPoller] inferencer: I have 0 tokens on my task list.
2011-05-02 18:14:13,351 INFO  [com.siemens.vcms.actor.common.JbpmPoller] authorizer: I have 0 tokens on my task list.
2011-05-02 18:14:13,352 INFO  [com.siemens.vcms.actor.common.JbpmPoller] streamingServerTransfer: I have 0 tokens on my task list.
2011-05-02 18:14:13,357 INFO  [com.siemens.vcms.actor.common.JbpmPoller] transcoder: I have 0 tokens on my task list.
2011-05-02 18:14:13,358 INFO  [com.siemens.vcms.actor.common.JbpmPoller] encryptor: I have 0 tokens on my task list.
2011-05-02 18:14:13,361 INFO  [com.siemens.vcms.actor.common.JbpmPoller] inferencer: runtime (ms) for tasklist: 26
2011-05-02 18:14:13,361 INFO  [com.siemens.vcms.actor.common.JbpmPoller] authorizer: runtime (ms) for tasklist: 26
2011-05-02 18:14:13,367 INFO  [com.siemens.vcms.actor.common.JbpmPoller] streamingServerTransfer: runtime (ms) for tasklist: 63
2011-05-02 18:14:13,368 INFO  [com.siemens.vcms.actor.common.JbpmPoller] transcoder: runtime (ms) for tasklist: 59
2011-05-02 18:14:13,369 INFO  [com.siemens.vcms.actor.common.JbpmPoller] encryptor: runtime (ms) for tasklist: 55
2011-05-02 18:14:15,167 INFO  [com.siemens.vcms.actor.common.JbpmPoller] streamingServerNoTransfer: I have 0 tokens on my task list.
2011-05-02 18:14:15,177 INFO  [com.siemens.vcms.actor.common.JbpmPoller] streamingServerNoTransfer: runtime (ms) for tasklist: 18
2011-05-02 18:14:15,550 INFO  [com.siemens.vcms.actor.common.JbpmPoller] middleware: I have 0 tokens on my task list.
2011-05-02 18:14:15,561 INFO  [com.siemens.vcms.actor.common.JbpmPoller] middleware: runtime (ms) for tasklist: 18
2011-05-02 18:14:15,983 INFO  [com.siemens.vcms.actor.common.JbpmPoller] metadataIngester: runtime (ms) for token 9148825717: 7348

fugix · June 6, 2011

it's in the first post, I just need the code he suggested implemented

i was asking for your new code where you tried to incorporate what webstyles gave you

kney · June 6, 2011

if I knew how, I wouldn't have asked it here

fugix · June 6, 2011

did you try his code

kney · June 6, 2011

I have this now

<?php

function parseTXT($data){
$file = file_get_contents("server.log");
$d_array = explode("\n", $file);
$lastWord ="";
foreach ($d_array as $line){
	$words = explode(" ",$line);
	$firstWord = $words[5]; // these are the substeps (like metadataIngester etc.)
	if($firstWord == $lastWord){
		// I don't know what to do here <============
	}
	else{
		$lastWord = $firstWord;
		if (strstr($line, "runtime (ms) for token")){
			$tmp_array = explode(" ", $line);
			$token = str_replace(":", "", $tmp_array[10]);
			$token_array[$token][] = $tmp_array[11];
		}
	}
}

foreach ($token_array as $key=>$val2){
	$token_array[$key]['total'] = 0;
	foreach ($val2 as $val3){
		$token_array[$key]['total'] += $val3;
	}
}
return $token_array;

}

foreach (parseTXT("some_file.txt") as $key=>$val){
echo "<h2>Token $key: </h2>";
echo "<table border=\"0\"><tr><th>MS to Complete</th></tr>";
foreach ($val as $key2=>$val2){
	if (is_int($key2)){
		echo "<tr><td>$val2</td></tr>";
	}
	else{
		echo "</table><br /><strong>Total time for $key: $val2</strong><br /><br /><br />";
	}
}
}

?>

fugix · June 6, 2011

do you receive any errors or you just dont know what to do from here? also your first word would be

$firstword = $words[0]

what exactly do you want to do if your first word equals your last?

kney · June 6, 2011

It is not actually the first word on the line cuz then I would get the date in my variable and i don't want that.

But i changed it a bit...

Now I almost have the result I want except sometime it doesn't seem to be working 100%

<?php

function parseTXT($data){
$file = file_get_contents("server.log");
$d_array = explode("\n", $file);
$lastWord ="";
foreach ($d_array as $line){
	if (strstr($line, "runtime (ms) for token")){
		$words = explode(" ",$line);
		$firstWord = $words[5];
		if($firstWord == $lastWord){
			// I don't know what to do here <============
		}
		else{
			$lastWord = $firstWord;
			$tmp_array = explode(" ", $line);
			$token = str_replace(":", "", $tmp_array[10]);
			$token_array[$token][] = $tmp_array[11];
		}
	}
}

foreach ($token_array as $key=>$val2){
	$token_array[$key]['total'] = 0;
	foreach ($val2 as $val3){
		$token_array[$key]['total'] += $val3;
	}
}
return $token_array;

}

foreach (parseTXT("some_file.txt") as $key=>$val){
echo "<h2>Token $key: </h2>";
echo "<table border=\"0\"><tr><th>MS to Complete</th></tr>";
foreach ($val as $key2=>$val2){
	if (is_int($key2)){
		echo "<tr><td>$val2</td></tr>";
	}
	else{
		echo "</table><br /><strong>Total time for $key: $val2</strong><br /><br /><br />";
	}
}
}

?>

This is what I get:

Token 3584819341:
MS to Complete
3422
5100
3424

Total time for 3584819341: 11946

you can't see this but the 5100 is also transcoder substep (as is the 3422, and I don't want it going through the same step)

and this is what i should get

Token 3584819341:
MS to Complete
transcoder: 3422
externalDRM: 3424

Total time for 3584819341: 6846

I also can't seem to get the names of the sub steps to display

kney · June 7, 2011

I changed in my code that when it's supposed to delete the double value, to change it to the string"double"

But 1 double he doesn't change..

Token 3584819341:
MS to Complete
3422
5100 <==  is a double substep
3424

Total time for 3584819341: 11946


Token 3604480140:
MS to Complete
5507

Total time for 3604480140: 5507


Token 3611033718:
MS to Complete
6297
2951
7308
3322
double

Total time for 3611033718: 19878


Token 3624140932:
MS to Complete
2989
2451
4487
2915
double

Total time for 3624140932: 12842


Token 3630694582:
MS to Complete
69829
7047
68111
double
double
double
double
double
double

Total time for 3630694582: 144987


Token 3643801742:
MS to Complete
2061
1503
3382
1781
double

Total time for 3643801742: 8727

kney · June 8, 2011

any help would be appreciated

Sign In

Read Text Files

Recommended Posts

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Link to comment

Share on other sites

Join the conversation

Important Information