[METHOD] Clone any site!! [With bonuses!]

Hey bhw members,

How are ya?

Hopefully banking hard

Ok enough talk.

I have read so many threads about how to clone a site or people asking how to do it.

Many have gone with HTTRACK and other software. Me personally, prefer a good and robust command line tool even if it has no GUI.

Well, many of you already know it, but for the ones that don't, I present to you WGET!

Basically, wget is:


And the best part is that it can be run in many OSs such as Linux and Windows (can't remember if it works on Mac).


First, you have to decide where you want to run WGET. It could be your server or your PC.

Of course if you are going to clone/copy a site to make it your own, you want to use your server instead of your pc.

First, let's start with Windows:

First, you will want to download the wget package from: http://ftp.gnu.org/gnu/wget/?C=M;O=D (or on: http://gnuwin32.sourceforge.net/packages/wget.htm), download on C:/wget and install on same directory.

Now you will have the following folders on C:/


Now you want to start the windows terminal, which you can do by doing either one of:

1) Win+R, type "cmd" without quotes and press enter
Or
2) Start=>Run=>Type "cmd" without qoutes=>Press enter.

Now you will have sth like:



And you are ready to go!

Now lets start the fun part!

First let's type and press enter:


From there you can use wget any way you want!

Now let's download a simple file using wget!

Just type:


And Voila! you just downloaded that whole site!

Now, let's go with Linux Servers!

First, make sure you have ssh or access to any terminal and connect to your vps/server/linux machine/etc.

Now that you are connected, make sure you have wget installed, by typing:


if it says sth that repository not found, simply install (if you have root access) with the command:


Once you are sure wget is installed, we can proceed with a very easy example of how to implement.

First, lets go to the temp folder by typing:


Now let's copy a good little site by typing:


Now you will see all of the downloaded files from that URL like:



Now go and clone any site you want!!!

Of course there are software out there that does this, but their usage is limited, so I suggest to follow the above steps


As a "bonus", I will clone/copy 10-20 sites from the first people that post what site they would like to clone!

If you need any help, DON'T PM ME!! Just post below and I will solve your problems asap! It's easier for everyone!

So that's it, BHW!

Enjoy cloning/copying your sites!

What can this do that HTTRACK cant?

Probably a good reason I block these sort of remote requests with .htaccess.

gonna try this....only we can do HTML sites?

CosmicSoundz said:

What can this do that HTTRACK cant?

Click to expand...

I like it better
And i can tell wget what i want, how i want it, where i wanted ALL in a single command lins

fmOzilla said:

gonna try this....only we can do HTML sites?

Click to expand...

It only requesting information, so yes, you'll get whatever the server gives you, not server side scripts.

File -> Save As if you just want a single page works well too. But yeah for entire sites good method wget is a great tool I use all the time on my elementary OS partition. For non-terminal peeps httrack is good and works.

heh, got paranoid for a second there, had to double check my file.

Not full proof, but makes me feel better knowing this ones on the list. Good reminder!

I don't understand what this does that copy+paste page sauce doesn't? Can I copy a whole website and its file structure?

Also using those commands on the window cmd prompt is so shit. Linux has a much better cmd system.

pxoxrxn said:

I don't understand what this does that copy+paste page sauce doesn't? Can I copy a whole website and its file structure?

Also using those commands on the window cmd prompt is so shit. Linux has a much better cmd system.

Click to expand...

Copy paste won't download any files at all actually, it'll simply transfer plain text to memory.

dinkish said:

Copy paste won't download any files at all actually, it'll simply transfer plain text to memory.

Click to expand...

I did this method and it just got the index.html an that's it. is there a way to get more? Or is it just god for informational website and not so much interactive websites?

pxoxrxn said:

I did this method and it just got the index.html an that's it. is there a way to get more? Or is it just god for informational website and not so much interactive websites?

Click to expand...

Did you type in both -r -m options? You should have downloaded everything that was "linked" internally. HTML/CSS/Images/Js/SWF.. I believe.

If your requesting something that is sent out from server side scripting (php, asp, whatever) you'll only get static html, this is just an automated way of caching sites keeping their directory\file structure intact.

Can you clone complicated sites like storenvy.com? Or charge a fee for it?

I don't see this being effective for AJAX either, as I believe that was to tie in server side scripting in a less obtrusive manner to the end-user.

looks awesome ... i would like a site

Do you know any way to also get .php files with the copy and not just the output?

You can't without server access. There's nothing lawful you could do to achieve this in reality.

Php is a server side script, meaning it's already outputted what you get client side, everything that serves prior to output is server-side.

can you please clone me facebook, thanks.