Markov chain generators source code, and their effectiveness

Daemon

Regular Member
Joined
Dec 31, 2009
Messages
209
Reaction score
79
I thought I would try and start a compilation of PHP Markov chain generators, and a discussion of their effectiveness. Below are a couple different generators - I believe the first is from YACG, and the second is from hxxp://www.haykranen.nl/2008/09/21/markov.
PHP:
function markov($content, $gran = 5, $num = 200, $letters_line = 65) {
  $combo = $content;
  $output = "";
  $combo = preg_replace('/\s\s+/', ' ', $combo);
  $combo = preg_replace('/\n|\r/', '', $combo);
  $combo = strip_tags($combo);
  $combo = htmlspecialchars($combo);
  $combo = explode(".",$combo);
  shuffle($combo);
  $combo = implode(".", $combo);
  $textwords = explode(" ", $combo);
  $loopmax = count($textwords) - ($gran - 2) - 1;
  $frequency_table = array();
  for ($j = 0; $j < $loopmax; $j++) {
    $key_string = " ";
    $end = $j + $gran;
    for ($k = $j; $k < $end; $k++) {
      $key_string .= $textwords[$k].' ';
    }
    $frequency_table[$key_string] = ' ';
    $frequency_table[$key_string] .= $textwords[$j + $gran]." ";
    if (($j+$gran) > $loopmax ) {
      break;
    }
  }
  $buffer = "";
  $lastwords = array();
  for ($i = 0; $i < $gran; $i++) {
    $lastwords[] = $textwords[$i];
    $buffer .= " ".$textwords[$i];
  }
  for ($i = 0; $i < $num; $i++) {
    $key_string = " ";
    for ($j = 0; $j < $gran; $j++) {
      $key_string .= $lastwords[$j]." ";
    }
    if (isset($frequency_table[$key_string])) {
      $possible = explode(" ", trim($frequency_table[$key_string]));
      mt_srand();
      $c = count($possible);
      $r = mt_rand(1, $c) - 1;
      $nextword = $possible[$r];
      $buffer .= " $nextword";
      if (strlen($buffer) >= $letters_line) {
        $output .= $buffer;
        $buffer = " ";
      }
      for ($l = 0; $l < $gran - 1; $l++) {
        $lastwords[$l] = $lastwords[$l + 1];
      }
      $lastwords[$gran - 1] = $nextword;
    } 
    else {
      $lastwords = array_splice($lastwords, 0, count($lastwords));
      for ($l = 0; $l < $gran; $l++) {
        $lastwords[] = $textwords[$l];
        $buffer .= ' '.$textwords[$l];
      }
    }
  }
  $output = trim($output);
  return $output;
}
And the second one:
PHP:
function generate_markov_table($text, $look_forward) {
    $table = array();
    
    // now walk through the text and make the index table
    for ($i = 0; $i < strlen($text); $i++) {
        $char = substr($text, $i, $look_forward);
        if (!isset($table[$char])) $table[$char] = array();
    }              
    
    // walk the array again and count the numbers
    for ($i = 0; $i < (strlen($text) - $look_forward); $i++) {
        $char_index = substr($text, $i, $look_forward);
        $char_count = substr($text, $i+$look_forward, $look_forward);
        
        if (isset($table[$char_index][$char_count])) {
            $table[$char_index][$char_count]++;
        } else {
            $table[$char_index][$char_count] = 1;
        }                
    } 

    return $table;
}

function generate_markov_text($length, $table, $look_forward) {
    // get first character
    $char = array_rand($table);
    $o = $char;

    for ($i = 0; $i < ($length / $look_forward); $i++) {
        $newchar = return_weighted_char($table[$char]);            
        
        if ($newchar) {
            $char = $newchar;
            $o .= $newchar;
        } else {       
            $char = array_rand($table);
        }
    }
    
    return $o;
}
    

function return_weighted_char($array) {
    if (!$array) return false;
    
    $total = array_sum($array);
    $rand  = mt_rand(1, $total);
    foreach ($array as $item => $weight) {
        if ($rand <= $weight) return $item;
        $rand -= $weight;
    }
}
Below are sample results, based on the content from the Wikipedia article for "Digital Cameras", and a granularity of 5. The sample code used to generate the results, and the results:
PHP:
echo markov($text, 5, 200);
This is not to be confused with the later cameras by Sony that also bore the Mavica name.) there are storage methods that use other technologies such as Microdrives (very small hard disk drives), CD single (185 MB), and 3.com/books?id=VYyldcYfq3MC&pg=RA1-PA16&dq=sasson+ccd+fairchild&ei=sDAFR-7YA46ApwLuvYijDQ&ie=ISO-8859-1&sig=FHJgLcIA9OrCgLxeJGpanGnMs8o.Analog electronic camerasQuestion book-new. p. For example, mobile phones often include digital cameras; those that do are sometimes known as camera phones. The choice of method for a given capture is determined largely by the subject matter. Line-scan-camera�based integrated systems, therefore are usually designed to streamline the camera's output in order to meet the system's objective, using computer technology which is also affordable.6 Line-scan camera systems o 1.0 Hasselblad H3DII-39 8,176 6,132 4:3 aspect ratio 50,135,232 50. 10, 1970. 2006-01-12.6, and 12. Typically, these cameras incorporate a nearly-silent leaf shutter into their lenses. Generally a contrast-detect autofocus system, using the image data from the live preview feed off the main imager, focuses the lens. For example, a 640x480 image would have 307,200 pixels, or approximately 307 kilopixels; a 3872x2592 image would have 10,036,224 pixels, or approximately 10 megapixels.com/news-updates/Polaroid%20Introduces%20the%20Polaroid%20PoGo%E2%84%A2%20Instant%20Digital%20Camera/. Eugene F. Canon demonstrated a prototype of this model at the 1984 Summer Olympics, printing the images in the Yomiuri Shimbun, a Japanese
PHP:
$order = 5;
echo generate_markov_text(1000, generate_markov_table($text, $order), $order);
900, Nikon D300, Nikon Coolpix 950 2,012 1,324 3:2 aspect ratio Actual pixel count". For examples 320 2,344 39.0 Hasselblad H4D-60 832 608 4:3 aspect ratio 7,077,888 7 Olympus E-500, Olympus E-500, Olympics, printed into them. Data coming from PDAs and no. For the majority have been three main methods for the image sensors, or three separate images to free storage protocol, and any pattern is a repeats. The data can quickly excessive pixels), and took 23 seconds to capture system of some offered to film, and the mirror flipping out of the Wikimedia Foundation * 12 References 1. ^ "Nikon D90, Nikon QV-1000C, designed as a cameras in technology for sensors (in terms is a CCD is actually take their filter printers. Captured for process � Fixer � Push processing � Depth of focus, and would seek out and focus on the Logitech Fotoman. It used in camera phone) * Privacy policy * What links Search Wikimedia Foundation, and at very high-end cameras are camera
Both have some problems with UTF8 characters. The first function seems to have results that are slightly less gibberish-like, but I haven't done extensive testing.

I've never used Markov generated content on any websites so I have no knowledge of its effectiveness. Any input in this area would be appreciated :)
 
i think markov chain we use to to rewrite content , i just harvested for building a synonmy databse , but the output you get is mostly garbage..
I think we what we need is a English major who can explain the us how we can use markovs chain to rewtite sentence..
Im..would like to throw a few suggestions...if you like,,
 
man you are my king! actually i searched for such code some days ago. i think there is a lof of power in this method. it just needs some optimization and a lof of reference data to produce good content. even if it sounds strange - big G wont see it as not by human written content. I'm sure it works. and if your content is worse more people will click the ads :D
 
Back
Top