Last year I found that migrating a WordPress blog is a difficult task. In March I redesigned and rebuilt davidnaylor.co.uk with the intention to create a more business orientated website. The site was to become a fully functioning website with a blog attached rather than running everything through WordPress. Because of this the site was built from new on a development area, this was fine until it came time to move the WordPress database.
As the new site was more than just a WordPress theme the new site could not be simply integrated into the old website, the WordPress database would need to be moved. Simply importing the exported XML file from the old website was not possible due to the size of the file. The site had amassed 8MB of data while WordPress stated an upload limit of 2MB.
After failing in my attempt to do the same with the SQL export file, I was left with splitting the XML file into 2MB chunks. This is a long and boring process.
Now the problem has cropped up again, with an even larger XML file and I have tried once again to find a solution.
So is there a solution?
None that have worked for me. I did find a few solutions that try to increase the upload limit to 8MB through the php.ini or htaccess files but these failed. My problem is that the XML file is larger than 8MB and when trying to see if I can up the limit further than 8MB there is no way of testing if it’s worked. If you’re lucky some methods will alter the upload limit shown in the WordPress admin.
Next I found a WP Splitter program on GBMINI but as it’s coded in Visual Basic 6 I could not get it to run correctly on my PC. In my case it seems the only option to move the database is to split the XML file manually.
How about automating it?
In WordPress 2.7 the development team added a one click upgrade feature for the whole WordPress install that complements the previously added plugin upgrade function. It seems to me that a database migration option could also be added. Given you have all the correct details for the database you wish to migrate from it should be possible to access the database and copy all the data across. This creates a quick and easy way to get the information that is potentially unhampered by the amount of information being transferred. However this could also be the case of me talking about something that although sounds simple is rather complicated, or in fact completely impossible to do.
A Quick Note
In writing this I considered the option of simply pointing the new site to the old database and when the domain points to the new site it will simply work. This is of course the easiest option, but gives you no opportunity to test the website with the correct database until the site is live. This is of course a potentially dangerous method as many issues can arise including plugins and themes specific to the new site becoming deactivated.
If you’ve come across a method that works then add a comment below.
15 Comments
If you have shell access to the server (as you’ve suggested modifications to the php.ini file, I’d assume you do) why not just import the mysql database through there?
Once uploaded to your server, mysql can import massive files quickly via the command prompt. Not sure about XML exports, but SQL ones like you have mentioned can be done effectively like that.
Douglas Radburn | douglasradburn.co.uk 19th January 2009 @ 9:51 am
@ scottmallinson the technicalities of that are a little beyond my understanding but I think I get the theory behind it. If I understand it correctly this would be a good way of dealing with the issue if you were the sole developer and tester, however would this not limit the viewing of the development site to a single PC, impacting testing and sign off through multiple persons.
kean 19th January 2009 @ 8:14 pm
Alternatively, you could create a local copy of the live database and any files, use your hosts file to ‘spoof’ your PC into thinking it is accessing davidnaylor.co.uk when in fact it will be accessing the local copy.
That way you will be able to setup and configure WordPress as you would in the live environment, make any amendments you need and then test it before making it live.
Scott Mallinson | www.scottmallinson.com 19th January 2009 @ 7:56 pm
@ kean Basically what I am proposing is that you change the Windows hosts file found on your PC, usually in C:\WINDOWS\system32\drivers\etc\hosts, to spoof your PC into thinking that 127.0.0.1 is davidnaylor.co.uk.
This solution is more suited to single developer projects, however it wouldn’t be much trouble to have the hosts file replicated across the PCs of those working on the project.
Alternatively, I understand that is it possible to perform a similar operation but at the server level. So on your network server, when a PC on the network requests davidnaylor.co.uk it points to a folder on the development server. This solution is more technical and I’m not entirely sure of the various settings that are needed, but it is possible.
I feel that the real problem is with WordPress itself. An installation will only work at the URL it is configured on, and with the database it is setup with, for example you can’t have a local WordPress installation working off a live WordPress database.
Scott Mallinson | www.scottmallinson.com 20th January 2009 @ 3:17 pm
The issue is most certainly that of the importing large XML WordPress files.
The idea of pointing to the old database rather than creating a new one does offer a solution, if not potentially complicated/dangerous when making edits to WordPress that would impact both current and development sites.
With the XML file though there seems to be no solution other than spliting the file, and although annoying and time consuming it appears the best solution to ensure a clean starting point if you are redeveloping a site from scratch. Though of course if you target specific id’s for categories etc you’ll have a larger task correcting these.
@ douglas I did try a simple export/import using SQL but it just bombed out, too many lines of code I think.
kean 30th January 2009 @ 2:54 pm
Not sure if you’re going a little off topic Scott. Yes, you can set it up like that and import all the data locally if you’re starting from scratch, but I assume Kean wants to combine the data. Kean seems to be suggesting that his ONLY stumbling block is file size here?
Douglas Radburn | www.douglasradburn.co.uk 30th January 2009 @ 2:27 pm
@Douglas Radburn My solution was to Kean’s suggestion where he considered pointing the new site to the old database, but wouldn’t be able to test it.
Sure it’s a long-winded way, but it would ensure an identical copy of the live database to work on for testing, and would prevent issues of mis-matched IDs for categories, posts, pages etc. as so commonly happens when importing a WordPress database.
Scott Mallinson | www.scottmallinson.com 30th January 2009 @ 2:40 pm
I assume you’ll have hit the PHP limits that would be imposed on the site for max upload/max execution time etc. Thats why if you have SSH access, you can get round all that.
Sorry, Scott, I misunderstood your intentions there. It appears as more of a different way of doing it rather than a resolution, although thinking out the box is always good when the you can’t find a solution!
Douglas Radburn | www.douglasradburn.co.uk 30th January 2009 @ 4:37 pm
@douglas exactly right.
Though if your talking about the WordPress XML upload then i’m not sure if SSH would be of any use. It would get the file onto the server but to use the normal WordPress import function you can’t import from the web as itonly allows you to browse your computer not input a URL.
Therefore you’d need a plugin or something to extract the information from the file on the server and input to the database and i’m not sure such a thing exists.
Kean 30th January 2009 @ 4:45 pm
It was mainly your comment about SQL to be honest rather than the XML. Although, I’m probably just looking at a work around if it usually exports as XML?!
Douglas Radburn | douglasradburn.co.uk 31st January 2009 @ 7:50 pm
hi,
if I understand correctly (i’m about to change my blog from one domain to another), i have to split my .xml file (18MB) in multiple parts…?
And how do i reconstruct it when i upload it to wordpress ?
Sorry for my english and my bad understanding (i’m french)
LOmiG | www.blomig.com 12th March 2009 @ 3:25 pm
If when you upload to WordPress your limit is 2MB you will have to split your file into 9-10 seperate files. With them seperated it is a case of importing each one by one and mapping any users to any already created. WordPress pretty much does the rest.
Spliting the files is also an easy but long winded process. When spliting the files it’s a case of deleteing as many posts as necessary to reduce the file limit to whats necessary I also leaving the common information at the top of the xml that contains your categories and tags in each file.
Finding the line number of the complete file and working out in this case 1/10th of that number gives you a guide about where to start deleting from, then just find the start of the next post and delete from there. Keeping a record of line numbers helps as you start deleting above and below the posts necessary for the next file.
Kean 12th March 2009 @ 8:02 pm
hi kean !
thanks A LOT for this great answer…this is exactly the level of answer i needed to go through the process. Well, it seems that i know now what my sunday will be like !
By the way, isn’t it very strange that all this stuff is still a manually thing ???
My WordPress installation says (7MB max) for the .xml file uploading.
THANKS A LOT !!!!
LOmiG | www.politique.blomig.com 13th March 2009 @ 6:10 am
I have the problem to change my blog for wordpress.com to a selfhosted wordpress install. It is a 2,4mb file with 800 entrys but all 100 imports it suddenly stops ? without error. Is there a way to split the file and import little files.. wordpress.com have no options to install a own plugin or advanced export options.
Mike | hosted4free.de 21st January 2010 @ 8:54 pm
@ Mike Usually if the file is too large you would get an error message saying this so there could be something else going on, but if we assume that the file size is the root cause then I’m afraid the only method I know currently of splitting the wordpress xml file is to do it manually. This will require some basic knowledge of xml or similar web language. It’s possible more automated systems are now available but other than the one mentioned in the post I’ve yet to come across them.
As you have only a 2.4MB file it may be possible you can increase the file upload size to 8MB using the php.ini file (other methods are available). But doing this will depend on the level of customisation your web host allows.
Kean 22nd January 2010 @ 8:38 am