how to group similar strings?

[hasanatkazmi]

Hi I am using PHP to build a system, the system will be provided with a number of sentences. I need to make an algorithm which can group the sentences on the basis of similarity e.g

 

I have following senteces:

"Musharraf vows that Nawaz Sharif will be arrested on his return because he is in exile because of the agreement he signed."

"Maria Sharapova seems to be very deterministic about her new ambition, she can win the title."

"The presedent of Paksitan: Musharraf is facing a lot of difficulties at Home, at one side Supreme court has ruled against him and on the other side Nawaz Sharif is comming back."

 

Now if I have a number of sentences (may be in thousands), I need to write an algo which can group first and the third sentence, and grop 2nd alone.  I need help in this? Do I need to study any specific technology for this which can help me in this?? pelase help he in the algorothm or give me sudo code if possible

[hasanatkazmi]

Please help me !@!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 

simple algo to group strings togehter

 

[Barand]

I'd do something like this. This example assumes the keywords are capitalised. For many sentences it would be better to define keywords in advance. You would also need to filter out keywords like "The"

 

<?php
$sentences = array(
   "Musharraf vows that Nawaz Sharif will be arrested on his return because he is in exile because of the agreement he signed.",
   "Maria Sharapova seems to be very deterministic about her new ambition, she can win the title.",
   "The presedent of Paksitan: Musharraf is facing a lot of difficulties at Home, at one side Supreme court has ruled against him and on the other side Nawaz Sharif is comming back."
);
$groups = array();
$keywords = array();

function keywordExtract($txt)
{
    $kw = array();
    $words = explode(' ',$txt);
    foreach ($words as $w)
    {
        if (strtoupper($w{0})==$w{0}) $kw[] = $w;
    }
    sort($kw);
    return $kw;
}

/**
* get keywords for each sentence
*/
foreach ($sentences as $txt)
{
    $keywords[] = keywordExtract($txt);
}

/**
* compare keywords and add to groups 
*/
$k = count($sentences);

for ($s = 0; $s < $k-1; $s++)
{
    $groups[$s][] = $s;                    // sentence goes in its own group
    for ($t = $s+1; $t < $k; $t++)
    {
        // any common keywords? 
        if (array_intersect($keywords[$s], $keywords[$t])) $groups[$s][] = $t;
    }
}

/**
* results
*/
foreach ($groups as $gno => $snums)
{
    echo '<h3>Group ', $gno+1, '</h3>';
    foreach ($snums as $sno)
    {
        echo '<p>', $sentences[$sno], '</p>';
    }
}
?>

[Barand]

PS. if I had spotted your bump (after only 20 mins) before I started to look at your problem, I would have skipped this topic completely.