How to spin an article to 70+% uniqe?

freller said:

I suggest looking up n-gram in Wikipedia and Representing n-gram language models for compact storage and fast retrieval in Google Patents and http://googleresearch.blogspot.co.uk/2006/08/all-our-n-gram-are-belong-to-you.html in Google Research - Bing have already stated that they index up to 7 word n-gram blocks and some of this stuff goes back six years so I have no doubt Google are ahead of that now.

Simple word spinning has not been sufficient for quite some time; to appear truly unique you have to alter both the sentence and paragraph structure. In other words, a complete rewrite. Using TBS or similar to spin words is just providing a fig leaf for little or no real benefit.

Click to expand...

ngrams change none of that (and I've read those patents and am familiar with them).

Take a look at those two sentences:

If you buy my product I'll be very happy.
Should you purchase my product I will be quite pleased.

Click to expand...

How do ngrams allow you to determine those two are the same?

And I don't completely disagree with you... that's why my automatic spinner would do a spin like this for that sentence:

{If {{you buy|you purchase|you get} my {product|item}|my product is bought by you} {I will be|I'll be|I am} {very|quite} {happy|pleased}|{I will be|I'll be|I am} {very|quite} {happy|pleased} if {{you buy|you purchase|you get} my {product|item}|my product is bought by you}}.

Click to expand...

Take a look at those two sentences:
If you buy my product I'll be very happy.
Should you purchase my product I will be quite pleased.
How do ngrams allow you to determine those two are the same?

Click to expand...

N-grams allow for the easy identification of common text blocks resulting from synonym swaps. If you additionally code in contractions (I'll/I will) - which Google do as they're really just another form of synonym - then your example is also covered.

Having said that, is Google actually bothering? Below Page 1 probably not, but at the very top of the SERPs I think they are making that extra effort to weed out spun text on the results returned.

As the next best thing to a manual rewrite I like your service the best, by the way, so I'm not against automation where appropriate (blog farms, article submits etc.)

freller said:

N-grams allow for the easy identification of common text blocks resulting from synonym swaps. If you additionally code in contractions (I'll/I will) - which Google do as they're really just another form of synonym - then your example is also covered.

Having said that, is Google actually bothering? Below Page 1 probably not, but at the very top of the SERPs I think they are making that extra effort to weed out spun text on the results returned.

As the next best thing to a manual rewrite I like your service the best, by the way, so I'm not against automation where appropriate (blog farms, article submits etc.)

Click to expand...

Basically what you are describing is Copyscape's algorithm.

Quick explanation of how Copyscape works for anyone who is curious (you can probably skip over this freller):
For the sentence "This is an awesome product that you should buy"
Copyscape will take the following phrases/ngrams: "This is an awesome", "is an awesome product", "an awesome product that", "awesome product that you", "product that you should", "that you should buy" (so each 4 word phrase) and it will search to see what matches there are for each of those phrases. If a certain percentage of those phrases all return matches from the same document, Copyscape will know that it is a copy.

So if you are spinning every other word (or even every 3 words), this algorithm will not be able to find common blocks, because there aren't any.

If you were to compare "Should you purchase my product I'll be quite pleased" to "If you buy my product I'll be very happy" you'd get the following phrases:
"Should you purchase my" (no match)
"you purchase my product" (no match)
"purchase my product I'll" (no match)
"my product I'll be" (match)
"product I'll be quite" (no match)
"I'll be quite pleased" (match)

There is only 1 match out of 6 (assuming it automatically makes I'll and I will the same), which is not enough for the article to come back as a duplicate - because there are other sentences on the web that'll also come out with 1 match (because a lot of those 4 word phrases are relatively common). Of course if throughout an entire article you have 10+ matches, then yes you will probably be caught by Google.

So if you are spinning every other word (or even 1 out of every 3 words) you will be unique to Google - really if you are unique to Copyscape you are unique to Google (as they are using pretty much the same algorithm, both of which are based on ngrams).

It could be that I am missing a part of Google's algorithm for detecting duplicate content, but I do not think that is the case. Either way, I'd like to hear your thoughts on this because you pay more attention to this and know more about these algorithms than most people whom I talk to about duplicate content.

Hard to have 70% spun and not look like total machine gibberish.

What Google can do, and what it does do, are fortunately two very different things.

Google fully understands synonyms as shown by its return of '~' values in the SERPs when you ask it. So, if you were to take an extreme (and probably impossible) example of a 500 word article where every single word was replaced by a single word synonym then Google would be able to identify the two articles as essentially duplicates. I say 'essentially' because I'm pretty certain Google works on two levels when it comes to assessing duplication.

At its most basic it does work, yes, like Copyscape, so replacing every second word or so (assuming 3 word blocks which is the level of granularity you should always work to) WILL return what is good enough to be thought of as unique content for Page 2 and below.

On Page 1 I'm certain that the selection process is more exacting though. To take the previous example where every word in a 500 word article is replaced by its synonym, the structure will still be the same as the original - same number of paragraphs, same number of sentences in each paragraph, same number of words in each sentence. That's a big giveaway.

Using inserts/callouts as in Article Builder only marginally helps because the whole of the article is still there - there's just some extra bits added in. It's the same as when you add Related Posts to an article - Google still sees the original article for what it is, without any hoped for benefit from the additional text.

There are many other tests that Google can, and does, apply for Page 1, but probably dispenses with for the lower pages.

I have ONLY ever had duplicate content consistently rate highly on the old Yahoo - Google and Bing have always been too clever for that where I didn't also do a lot of backlinking. On my highly ranked authority sites I USED to get away with it with them as well, but not since April this year. Ever since then I've been progressively losing places for duplicate and near duplicate content.

'Ultra spun' articles where the sentences remain the same, but the article structure changes, are also not the answer. Try doing some exact match searches on sentences from Article Builder and The Leading Articles output - we're talking many thousands of returns in almost every case, which is interesting in itself - they may not rank but they ARE being indexed.

To make those services really worthwhile you also have to spin the word structure as well as everything else, and you're then talking vast amounts of time and effort. Hardly cost effective.

To get to Page 1 you MUST completely rewrite any article. However, for feeder articles I'm personally convinced that synonym swaps are probably sufficient.

One final point - have you ever wondered why your tag, category and author pages often rank higher than your actual articles where you use a lot of duplicate content? It's mainly because the engines see those pages as having unique structure and/or semi unique content (usually abstracts in the case of category pages) compared to your other content (there is some benefit for categories from linking from the front page, but I've often found front page articles to not rank as well as the category pages, particularly on Bing). That particular observation has strongly guided the way I structure pages now.

You're probably gonna neg rep me for this...but have you tried pressing spin twice? It works honestly!