Erin Clerico

I think I am ready to start the official testing phase - the shakedown is nearly complete. What’s the difference between shaking down new software and testing it? Before the hunt for the real bugs in a semi-biggy process like this, one must fire it up and let it run top to bottom on a few sites and see if any nuts or bolts fall out. Nail the easy bugs first. The hunt for the fun bugs, the tough ones is when the real testing starts.

The process is running front to back on a large, complex site as I blog this.

Lots of bugs have been whacked this last weekend, the converted site product still has a few too many warts to declare then end of the shakedown period.

Tomorrow I will take on a second site, another news-post oriented site. My feeling is that this process will do a better job on news-post, or blogging oriented Manila sites.

Brochure type Manila sites - with a deep site structure for example, won’t convert as well in the first version. I will try out the process on a brochure type site before I’m done.

No one ever told me porting Manila to WordPress was going to be this much fun!

I hit a limit on my method of fetching Manila Gems (Files in later versions of Manila) and posting them proper into a WordPress site at around three megs. The script hit a file of 8 megs and I discovered that Manila would not parse the xml-rpc request for a file that size.

I elected to have the WordPress site try to fetch the gem, and then post it into the WordPress site directly.

My script calls the WordPress site via xml-rpc, and gives it the URL of the gem. Wordpress processes this command and then returns the new URL - which my Manila script will store along side the original url so we might be able to forward traffic to the new address of the gem. This is all via a custom api I have put together specifically to assist the site conversion, in case your wondering how I get WordPress to do these unnatural things.

As simple as all that sounds, it’s less than attractive hack (it does work however) - so I think I’ll call this one a tie and move on.

I have selected another Manila site I host, www.londonpoolscampaign.com as a pre-testing test victim and found quite more than a few of member problems. in addition to the remnants of some sort of mass member creation fiasco (lots of consistently malformed member logins), it looks like there there were tons of members created with no record of a login. This is the result of Manila site owners enabling a poorly conceived commenting system called ‘Radio Hosting.’

This feature invited tons of abuse by comment spammers, and so I am inclined to leave these members and their comments behind, as a site that has been abuse can run in to tens of thousands of members like this. Please feel free to shout me down (okay don’t really shout, I’m a sensitive kinda guy) on this point.

Also I have opted to leave out members who have not logged in for two years or more. This may indeed be harsh - again please feel free to debate this with me. Any content posted by one of these expired users, will be the property of the admin on the new WordPress site.

So here are today’s changes to exporting members from a Manila site to a WordPress site:

1. You have a login, but have never logged in? Your out.

2. You have not logged in for the last two years? Your out. Your posts become property of the new admin.

Update: see the new chart ‘A close examination of exporting Manila members to WordPress‘ that I just posted.

Cecil re-enforced the point about not having a good Manila glossary to render new shortcuts from the new WordPress (WP) URLs for each story discussed earlier. He’s right, and this has brought about a major shift in the process here. This is important as this is the mechanism that will make sure your links in all your content that point to your own site sill still work after the conversion.

On one hand I want a table of old Manila URLs matched to new URLs, and the only way I can see to do this is to plow thru the Manila site in the exact order it was created - message one then message two then three etc. The trouble is that the glossary is not of a chronological nature. And he’s right, users go in and mess with it all the time.

So here’s the new plan - please hold your nose and remember, reshaping a large amount of complex, interrelated data in to a new schema never designed to house it is by nature a kludgey, messy operation - best to accept homely solutions (and refer to them as ‘novel approaches’) and just move on.

I’ll make one complete pass over all stories, home pages, images, GEMs and news items, accumulating new WordPress URLs for each resource as I go.

When that is complete, and I know the whole truth about every URL in the old glossary and have a new glossary against which I will re-render every Manila story and home page and then post the result as an update to each already existing WordPress page. Remember, this should only be necessary for Manila stories and home pages, possibly discussion posts as replies to home pages and stories since Manila ‘news posts’ do not render shortcuts or Manila macros.

Only a little easier said then done, and I know what your thinking but remember — this a ‘novel approach…’

I updated the flowchart to document this insanity - should have the code actually doing this shortly.

I have started up the debugging process on the Manila to WordPress conversion process.

So far we have all members from the Manila site being created remotely, complete with WordPress style logins - which can’t be e-mail addresses. My sollution has been to take the Manila e-mail login, take the ‘@’ sign and any other non-alphas like periods etc out - making John Doe’s Yahoo address into ‘johndoeyahoocom’ for John’s WordPress login. If John was a Managing Editors, or some other level of editor, hi role on the WordPress site is set at the same operational level.

We also have all Manila New Item Departments being created on the target site, so when we post all the Manila News Items they will have the same categories on the Wordpress site.

Manila Gems, if available from the Gems static server, are being uploaded to WordPress properly and their old URLs are stored with their new WordPress orient URLs so we can do some re-direction magic on the new server. This is nice, keeps old bookmarks, links and search engine referrals working, no breakage.

More to come in the next few days. I may be able to accept a few test users to get some feedback very soon. Stay tuned.

Another day another quagmire in my quest to convert Manila sites to WordPress… Manila users can, and often do post item in to the discussion group as top-level threads, and then subsequent replies. Pretty typical set up for a forum type program. Trouble is WordPress is not this type of animal.

I’m afraid there is no similar shape in WordPress land - content in WordPress are pages, blog posts, comments to pages or blog posts, attachments, tackbacks… but nothing I can see that resembles just a plain old discussion forum type top-level thread.

Possibly, I could simply create a custom news category on the target WordPress site titled something like ‘Manila Discussion Group Item’ and post top-level threads there. Trouble is with that approach that discussion group posts would sprinkle in among regular WordPress style home page published posts. This seems unnatural and confusing.

In the spirit of the k-i-m-c-s approach (keep-it-moderately-complex-stupid) that is my nature, I think I’m going to drop support for this type of content. If I get strong push-back then I’ll wander into this thicket again and scare something up to export this kind of content. I guess in the meantime I’ll make this fact plainly clear in the disclaimers on this process.

As I have developed the new Manila site export mechanism the have been a few little brick walls I needed to break thru to keep the process moving forward.

In my first pass over the requirements of this project I tried to do as many proof of concept runs to try to get a handle on what looked like the most difficult tasks in actually trying to export a Manila site entirely via XML-RPC to WordPress.

At the top of the list it a seemed was the fact that the call (getting techie here) ‘metaWeblog.newMediaObject’ did not seem to want to work at all with WordPress. So of course this became the very first task I undertook, since it appeared to be a show-stopper if I couldn’t lick it.

After a days or so of moderate hair pulling I finally pinpointed the problem within the Manila server.

Once I corrected the bad code it fired right up on the WordPress end and thus the chief obstacle as far as I could see at that time had been knocked off.

A move from Manila is long overdue for Weblogger. I’ve been working for some time on putting in to place everything I will need for a completely new Weblogger.com.

So many things will be better - lots of new billing automation to provide faster sales and configuration for our clients, new blogging platform, better support features - basically everything will be new from the ground up. We will still be supporting Manila hosting for as long as any of our currenty paying customers want it. I’ve been blogging about a conversion process to allow Manila users to move to WordPress, and I will offer the service for all Manila users packaged with annual WordPress hosting, so everyone wins.

The home page will soon have a new look and new marketing to reflect our transition. More on that soon.

In my previous post I described a one-brick-at-a-time approach for moving the content of a Manila site to a WordPress site. Just how am I planing on doing this you ask?

I’m going to use yet another Dave Winer invention, XML-RPC. This is a pretty heavy concept for non-programmers, so I’ll try to explain what this means in simple terms.

When you load your blog into your browser and log in to make a post - you are really just feeding some inputs (the title of the post, the post, links etc) in to a program, order ‘code’ that saves the post in to a database. When you view your post, it has been read from a database - fed thru some other part of your blogging program to display the post on your screen.

Most blogging platforms have a series of ‘hooks’ that activate some of the same code that you use through your browser. Programmers often refer to these ‘hooks’ as APIs (Application Program Interface). This means I can write a program and use it to post to my blog via it’s APIs - that is I don’t have to use a browser - I could script an automatic process to make a post to my blog every time my laptop wakes up, for example.

This is where XML-RPC comes in. WordPress has a set of APIs - which use a standard called XML-RPC to carry the inputs and return the results to my custom laptop opening program.

Now one more leap here - I can program a Frontier/Manila server to read a Manila post out of the Manila database - and post it to a WordPress blog automatically. If I set up the program to ratchet thru each Manila post - make some decisions about the nature of the post (a news post, a page, an image etc) - package it up according to XML-RPC and WordPress API standards and posting it to the WordPress blog I want to move to.

This is how I plan on moving a Manila site brick-by-brick to a new WordPress site.