1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Importing Huge Amount Of Post To Wordpress?

Discussion in 'Blogging' started by Chicilikit, Jan 24, 2014.

  1. Chicilikit

    Chicilikit Senior Member

    Joined:
    Dec 21, 2010
    Messages:
    873
    Likes Received:
    153
    Hello, one of my methods needs importing huge amount of post. Right now for example I need to import 40K posts. I do it via csv, but I have great problems with that. No importer I have so far tried made it past few hundreds post. After that it always freezes and I have to start again. I have tried to cut the file in smaller parts, like 3K posts in one file, but it cant import even such small parts. Right now I get most like 600 posts from one import and it is second day I import the posts and still have not finished.
    What do you think causes this? Not enough memory my wp site gets from hosting? I have already raised it via php.ini to some crazy numbers, it looks like this now:

    memory_limit = 1024M ;
    max_execution_time = 60000 ;

    Well, it does not seem to help much, so I do not know much what else can I do. I think about paying better hosting, but i need to know first if that is really the issue, if its even possible to import such large numbers into wordpress. Thanks for help.
     
  2. innozemec

    innozemec Jr. VIP Jr. VIP

    Joined:
    Aug 19, 2011
    Messages:
    5,289
    Likes Received:
    1,799
    Location:
    www.Indexification.com
    Home Page:
    first disable pinging as i think it pings on each imported post and that makes big lag

    also try to disable any post related plugins that could be doing work on each new post and also lag the import process

    look at your apache logs what exactly the error is when it freezes up and by that you can judge what settings you need to change
     
    • Thanks Thanks x 1
  3. netmatrix

    netmatrix Regular Member

    Joined:
    Mar 21, 2010
    Messages:
    303
    Likes Received:
    83
    Location:
    Midst of the Matrix... Exploiting Glitches
    WP Import Pro is supposed to be able to import thousands of posts in one upload. I've never used it myself.
     
    • Thanks Thanks x 1
  4. Chicilikit

    Chicilikit Senior Member

    Joined:
    Dec 21, 2010
    Messages:
    873
    Likes Received:
    153
    Thanks a lot guys for your help, gonna try it and will let you know if anything helped.
     
  5. Gogol

    Gogol Elite Member

    Joined:
    Sep 10, 2010
    Messages:
    3,063
    Likes Received:
    2,872
    Gender:
    Male
    Import batch by batch. That would help.
    Use set_time_limit(0) on the top of your script. That will prevent your script from getting timed out.
     
  6. dionysus

    dionysus Junior Member

    Joined:
    Dec 15, 2008
    Messages:
    104
    Likes Received:
    31
    This is the plugin I use for all csv imports

    http://wordpress.org/plugins/wp-all-import/

    I've used it to create affiliate sites from datafeeds, import thousands of posts, and set them up for dripfeed. Stupid easy.
     
  7. Chicilikit

    Chicilikit Senior Member

    Joined:
    Dec 21, 2010
    Messages:
    873
    Likes Received:
    153
    Wp all import is really good, I do not get any memory problems with it, but for some reason it messed up html code. < shows like ;quot and so. Do not know how to fix that :(
     
  8. dionysus

    dionysus Junior Member

    Joined:
    Dec 15, 2008
    Messages:
    104
    Likes Received:
    31
    There is an option to not convert HTML entities...
     
  9. Chicilikit

    Chicilikit Senior Member

    Joined:
    Dec 21, 2010
    Messages:
    873
    Likes Received:
    153
    I know, it does not seem to help.
     
  10. nottensed14

    nottensed14 Newbie

    Joined:
    Jan 27, 2014
    Messages:
    15
    Likes Received:
    1
    It could be a server side timeout, check with your hosts as they may be able to make suggestions.
     
  11. Gogol

    Gogol Elite Member

    Joined:
    Sep 10, 2010
    Messages:
    3,063
    Likes Received:
    2,872
    Gender:
    Male
    are you missing htmlentites() ?