Paid Review: Data Deposit Box Continuous Online Backup

This is my first "real" paid review through the ReviewMe service. Today's topic: easy online remote backup! The product in review: Data Deposit Box from Acpana Business Systems Inc., a Canadian company based in Toronto.

Um, Where's my Files?

I'm pretty meticulous about backing up my data. When I was a happy-go-lucky kid in high school, I remember working all night on some paper or program, only to accidentally LOSE HUGE TRACTS OF IT because I forgot to save. Or I would accidentally save over the wrong file, destroying a critical fragment of my personal history. Small disasters of this kind would occassionally crop up even through grad school; I might be working on some Photoshop file on a System 8.x Mac, and then the entire computer would lock up for some unknown reason. Hours of work lost, much cursing and swearing ensued.

So I evolved. I have an automatic "save" habit that kicks in everytime I pause to reflect. I automatically save new versions of files, with new filenames, using a versioning system. I archive and copy across multiple disks on multiple computers when possible, and burn to CD. I use version control software. I've gone through great lengths to separate the operating system, applications, and data onto physically separate drives; that way, if I have to restore my operating system due to some catastrophe, my DATA will not be erased in the process.

Frankly, I'm kind of a nut about data archiving and redundancy, but normal people have better things to do with their time. Occasionally, one of them will ask me how they too can not lose all their mail every time they upgrade to a new computer, or perhaps they've experienced for the Nth time some horrible loss of an important file due to a hardware failure. I tell them what I do, and their eyes glaze over. Most people find this to be a chore, but I actually like it, for some perverse reason. It's my version of gardening, I guess.

Which is why I found the prospect of reviewing Data Deposit Box interesting. It claims to be designed specifically for non-technical people, secure via its on-the-fly data encryption, and affordable at 2 bucks per gigabyte a month, paying only for what you use. While I don't have a particular need for it, I know lots of people would rather have a service take care of this for them. Let's take a quick look.

The Basics

Data Deposit Box (DDB) runs on Windows PCs, and installs as a program that monitors certain folders of your choice. Whenever a change is made to the contents of that folder (say, your "My Documents" folder), DDB detects that and then uploads the file to their encrypted server over the Internet. The cool thing is that once you tell DDB which folders to monitor, you don't have to do a darn thing except leave your computer on long enough so it can do its thing. If you're in the habit of turning off your computer every time you are not using it, then this program will probably not work for you. I leave my computer on overnight so it can run its daily virus check, so it works well with me.

Strapping In

Because DDB runs in the background, I was particularly concerned about how well behaved it is with respect to other applications. I'm constantly running big apps like Photoshop, Illustrator, Flash, and Dreamweaver with Excel, Thunderbird, Firefox, and Word open in the background. I am very sensitive to any program that gets slow and bogs the computer. There are several things I check for when installing a new system-level utility like this.

1. How big is the installer? Smaller is always better. In this case, the installer was about 3.83MB, which is fairly small. Compactness is often a sign of good and lean software engineering, though it's no guarantee. If I'd seen anything greater than maybe 7MB, I would be suspicious...an overzealous marketing department perhaps loading up the software with giant images and video files in an attempt to make their product more consumer friendly.

2. Create a System Restore Point. Windows XP has the ability to create a "snapshot" of your system before you install something. You can go to PROGRAMS -> ACCESSORIES -> SYSTEM TOOLS -> SYSTEM RESTORE to access the tool and create a restore point. After I'm done experimenting with this, I'll do a restore and rollback to the state my system was in before.

3. Turn on Performance Monitoring. This is an computer administration tool, available at CONTROL PANEL -> ADMINISTRATIVE TOOLS -> PERFORMANCE, that lets you monitor some of the inner processes of your computer. The ones I was watching was % Process utilitization overall and by the Data Deposit Box program itself, and also network bandwidth used. If DDB was a process hog, I'd see this plotted in real time on my monitor.

4. Turn on Process Monitoring. I just like to know what's going on, so I installed Winternals Process Monitor to watch as the program did its various things. ProcessMon tells you secret things about the computer, showing you what programs are doing at the operating system level. Perhaps I was being a little bit overzealous in my monitoring, but hey, I'm a nerd.

Then I ran the Installer, and braced myself.

The Particulars

The first thing the installer asks (after making you accept the End User License Agreement) is what folders to watch. By default it will monitor:

• My Documents
• Desktop
• Favorites
• Microsoft Outlook
• Microsoft Outlook Express
• Windows Address Book

That's a pretty good default list for most people, and you can modify it later if you wish. Since I keep all my data elsewhere, I unchecked the defautls set the folders manually after installation.

DDB installs a task tray icon that allows you to pull up the main dialog. Here's some screen shots:

The main dialog box. The main options I used were OPTIONS (to select which folders to back up and how) and RESTORE (to check that it really worked).

Here's the settings for the program. You can see you have some options on the number of versions to save per file (you can have up to 21, though I was unable to determine how to access any particular one) and how aggressive to be in terms of bandwidth and CPU hogging.

This is where you set which folders to watch. When a file changes, DDB will start uploading the changes in the background using your Internet connection.

In case you're wondering, the Advanced tab allows you to set Proxy Internet settings (if you have a proxy server that sits between you and the Internet, like at work) and where to store temporary files.

Uploading Files

After I set the folders to watch (about 35MB of files), I did some reading while watching the uploads out of the corner of my eye. The first thing DDB does when it activates is do a version check of all the files under its care; this can take a while. Then, it uploads the changed files to the server. On my cable modem connection, my upstream rate (i.e.: the fastest I can upload) is about 50K a second, so 35MB takes a bit of time to sync up. This is the kind of thing you'd probably want to run overnight, or if it's running during the day you would probably want to set the Bandwidth/CPU Allocation on the Options dialog to less than 100%.

The files are all encrypted, though I'm not sure what key it uses to encrypt them.

Restoring Files

After you've uploaded the files, you can click on the RESTORE button and get a list of files you can recover back to your computer. This seemed to work. As I mentioned, I couldn't determine how to access the versions of a file. I edited a text file a few times to see how quickly DDB would pick up on the changes and upload it. I wanted to restore the very first version, but didn't see how to do it immediately. It might be a buried option somewhere.

Sharing Folders and Files

Once your files are uploaded, you can choose to share them with people. This is not a file sharing service for MP3s, mind you (this is expressly forbidden in the EULA), but if you have a client that you'd like to provide a file to, you can do that on either the folder or file level. You can do all this through the online interface.

This part of the experience, actually, was a little less robust than the rest of the application. When you delete files, the tree view showing all my folders didn't update, so I was unsure if anything actually had happened. I forced a page refresh to see that it worked. A couple times I saw a database connection error, which was less than reassuring. The interface could use a little smoothing out, I think, but it was otherwise pretty usable.

One other note: the file sharing URLs that the system generates are ludicrously long. They're prone to wrapping in an email message, which creates problems when distributing links to people.

The Experience

Because the ReviewMe terms allowed only 2 business days to write this, I can tell you about long-term stability of the program. However, I can tell you that the experience was not bad. I didn't experience processor issue or problems with the installation, and the program did seem to work as advertised. I didn't notice anything that would make me uninstall the program immediately, which is actually kind of unusual. Any of the following are grounds for banishment from my system:

• bloatedness relative to function
• sluggishness
• ugliness
• excessive marketing
• bad UI
• excessive bugs
• unclear function
• unclear feedback about operation

I didn't experience any of that, so I would say that the desktop component was surprisingly good. There are a few confusing UI spots, but it behaved well and seemed to do the job. The help button isn't context-sensitive, for example...that's a minor quibble. I was happy that my system didn't get bogged down with this running continuously. And, it's not ugly or filled with questionably-useful graphic imagery, and I didn't get cross-sold on "other wonderful products you might be interested in".

I was a little less impressed with the robustness of the Internet side of things. It looks like a solid "Web 1.0" application, but seeing an ODBC database connection error on the "My Data" page does not fill me with good cheer. The experience is almost there, and may be above average for services of this kind, but my daily web experience revolves around Web 2.0 apps like BaseCamp, and they are shockingly robust. The bar has been raised!

What about the Cost?

This isn't a free service, so you have to weigh the cost/ease of use between their service and just buying a big external hard drive.

Data Deposit Box Pros: Works in the background, and it behaves with the rest of the system (at least, in my limited experience). Don't have to think about it, and data is sharable with other people. Backup is also offsite, so if something happens to your office, your data is safely somewhere else.

Data Deposit Box Cons: Costs money, online interface could use some tweaks, takes a relatively long time to upload data compared to using a local hard drive.

There are some other uses I can think of immediately:

1. I could set something like this up for family members under a single account (there can be as many users as you want). No relative of mine would ever lose their email again.

2. If you think of the service as file hosting, $2/gigabyte isn't that bad compared to regular web hosting. Factor in the ease of backup and sharing, and it seems quite reasonable.

So while I can't make any long-term assessments on the product, my initial impression is quite favorable. Certainly worth a look. It lasted on my system way longer than Adobe's Version Cue software, which was so slow I thought my computer had crashed. All that process monitoring software didn't raise a single red flag in my brief session.

There's a 14-day free trial available. I think I'm using it now. That reminds me, it would be nice to see some kind of feedback about how the free trial works before you hand over your credit card number...that is offputting enough for me to not even want try the service.

Again, this was a paid review booked through ReviewMe.

» Link: Data Deposit Box

Stupid Private Posts & the Compact Calendar

I was curious why people were reporting having trouble accessing the Compact Calendar. Usually, my site access problems stem from PHP timing-out while WP-Cache is attempting to build the page, which means that nothing shows up. Usually I clear the cache, and the problem goes away.

This time, though, it was a user error on my part; apparently I'd set the Private flag on that post while making some update. WordPress then continues to show me the post because I'm logged-in to the system, but everyone else sees the message sorry no posts found. Because of the clicky nature of the post editing window, which took a small usability step backwards in 2.0, it's not difficult to accidentally click and set the post status to private without realizing it.

Anyway, the Compact Calendar is back online...I apologize for the trouble in accessing it. And for WordPress users, here's what I did to fix it so this doesn't happen again. WARNING! Geek talk follows!

Making the Invisible Visible

The problem is that I can't tell as a logged-in user that a post is marked private. This is entirely because my aging WordPress template isn't coded to display this information. It's been frankencoded out of bits of the old WordPress 1.1 template with MovableType's old CSS. So the solution is to add some conditional logic to check the following:

1. Is a private post being displayed?
2. If yes, then display something like "PRIVATE POST " somewhere visible on the page.
3. If no, display normally.

The challenge is now to figure out how to modify my template to do this, which means that once again I get to dive into the WordPress API. Which is incompletely documented. The way that I've cobbled together my bits of functionality is by studying other plugins and templates discovered through dilligent googling, using search terms that are descriptive of the problem ("wordpress show private posts") and are also a bit technical ("template is_private"). The "is_private" search term is based on a guess I made about what function WordPress might have to help me detect if a particular post is private or not; there are several other 'is_this' and 'is_that' functions that are built-in to the system. Read on.

Checking for a Private Post

There are a number of special functions that you can use in your WordPress Loop to conditionally display information. The Loop is the PHP code that actually pulls your articles one-by-one from the database and displays them in vivid HTML; if you want to modify the appearance of a post based on its category, you can use the is_category() function to check whether it is in a certain one.

There's a whole bunch of such functions available, and I've never found a complete and comprehensive list of them in one place, until I stumbled upon PHPXRef. There's a complete WordPress function list available (click the functions tab at the top of the screen) that lists everything that I need to see for the latest version of WordPress. Alas, there is no additional documentation other than the list, but since WordPress is coded with fairly clear function and variable naming conventions, a programmer with direct experience with the application can scan the list and get an idea of how the application operates just by seeing how elements are named. It's not unlike being able to walk into an office and read, just by looking at the arrangement and details of the space, how things get done (or not).

The specific function I was looking for was something like is_private(), but SURPRISE...there IS no such function! Fortunately, someone else already figured this out and posted a solution. The code just queries the WordPress post directly and checks the post_status flag. I didn't know how the WordPress posts were organized, so this saved me a lot of digging. I took the function and dropped it into my template directory's functions.php file. This makes the function available for use in my template (and ties it directly to my template, which makes it more portable from installation to installation).

Now I can make the changes to my template. There's a line of code in it that looks something like this:

<?php
    if ( is_private() )
      echo "<span style='color:red'>PRIVATE POST</span>";
?&gt;
Filed under &lt;?php the_category(', ') ?&gt;

The conditional IF statement will prepend PRIVATE POST (in red letters) in front of the normal "Filed under [category]" bit of text, but only if it's a private post. For regular users, this will never happen. For ME, I should see the warning signs pretty clearly next time I bungle the privacy setting on a post.

Going Further

There are 30-something other simple is_something functions that you can explore at PHPXref, along with everything else. Unfortunately it's not exactly documentation. I find it useful for just seeing the "big picture" of function availability. If I want to know what the function does, I have to click on the function name, then look at the source code to divine what it actually does. It's not a bad way to learn PHP though, by modifying a piece of software you use everyday.

Here's an example of exploring the reference: I was scanning the list and the is_preview() function caught my eye. So I clicked it, then clicked on the defined in line # link on the next page, and this took me to the place in the WordPress source code that defined it. You can follow around such links until you find some comments that tell you what the hell it's supposed to do; in this case, the pertinent comment in wp-include/classes.php was if we're previewing inside the write screen. Now I know what this function does.

There's a lot of tracing lines of logic and following declarations, so this technique won't work for the novice programmer who's just learning how to code. For more experience programmers, it's a pretty quick way to explore the code without a heavy-duty integrated development environment (IDE) capable of cross-referencing functions, variables and classes. Doing this on the web is actually a little more convenient for me, since I don't have to worry about messing up my WP source files by accident.

Another cool thing: the PHPXRef site also covers other blogging and CMS systems, so this could be a very useful resource for understanding the architecture of those systems using a "ground-up" approach...which is really your own option when the official documentation fails to provide that picture for you.

So that's that.

Moving Wordpress Part I

First Post, The Second Time Around

Ok, I'm about to embark on my server moving adventure! I'm moving my entire davidseah.com domain from pair.com to futurequest.net, and I may do it yet again with MediaTemple in a few weeks to try them out. Here's my live move notes, which should come in handy later.

Moving the Database

Here's what I did to move the blog:

• Disabled Mint stats package on all pages so it doesn't access the database. I'm not going to be using it on the new server, will stick with Google Analytics for a while and see how that goes.

• Changed Basecamp's File Upload to use a different server. There are still some problems with old links though, so I'll have to add a redirect later.

• Moved all important files over (tar gz'd, ftp server-to-server). Had to nuke some, because the disk space limits on FutureQuest are pretty limiting (another reason to consider MediaTemple).

• Recreated mailboxes on the new server, including the dozens of forwarding rules I use.

• Locked out comments on the old server by modifying wp-comments-post.php to die() with a "sorry, posting comments is disabled" message.

• Used mysqldump wordpress_db --opt > backup.sql to create a backup file. I had to use the --compatible=mysql40 switch, because the version of MySQL on FutureQuest is an older one.

• Created database on the new server for WordPress, to restore the backup into.

• Used mysql wordpress_db < backup.sql to import the data. No errors! Yay!

• Used PHPMyAdmin, which I had installed earlier, to drop the tables I didn't need from wordpress_db. These were mostly old stats from Mint and SpamKarma. Also ran REPAIR and OPTIMIZE just for the hell of it.

• Edited the wp-options table in the wordpress_db, specifically the siteurl. I pointed it to the temporary URL for the new server. Then I went to my sites admin panel and set the blogurl in general options to the same. Why do this? If I don't, then clicking on internal links sends me back to the OLD blog, which makes it difficult to test this one. Eventually I will change them back to point to davidseah.com.

• I had to restore the auto_increment and eliminate spurious default values of '0' for each table's id, otherwise I got posting errors like Duplicate entry '0' for key 1. There's a bug in mysqldump where it does not preserving auto_increment when the compatible=mysql40 flag is set. All of the wp_ tables needed the default '0' removed, and the 'auto_increment' flag set. I used phpMyAdmin to reset the value.

• Deactivated / Activated Plugins, just for the heck of it.

Testing the Site

I disabled wp-cache and clicked through the following on the new server:

• Checked blog categories one-by-one
• Checked the "About Dave" links on the sidebar
• Checked all the .htaccess redirects I have, to make sure they worked
• Changed absolute links from http://davidseah.com/archives/... to just /archives/...
• Checked FAlbum links and htaccess redirects
• Checked comment links from the sidebar
• Checked link to the New Media Group posts
• Checked public RSS feed links
• Checked private RSS feed link for Feedburner
• Checked Google SiteMap Plugin
• Checked robots.txt
• Checked The Printable CEO download links
• Checked links in the footer
• Checked services that might be referring to graphics on davidseah.com (forums, etc)
• Posted a comment...works, after restoring the auto_increment and default value issues I mentioned above.
• Posted this post.
• Scanned the directory listing on the old server to see if anything else jogs my memory...nothing that has to happen right away.

The only thing left to do is switch nameservers and cross my fingers.

Down for the Weekend

I'm going to be moving davidseah.com to a new server starting Friday (October 20). I'm currently using Pair Networks, which has been great. I'm moving to FutureQuest, which has as gold-plated a reputation as Pair. The reason I'm moving is primarily because I'm starting to exceed the "maximum transactions per minute" limit allowed by Pair's database servers during peak times. I'm also curious about FutureQuest, which I've heard good things about, and how the heck one moves a Wordpress blog in the first place.

My checklist looks like this:

• Recreate mailboxes and forwarding aliases on the new server.
• Copy files over to new server via server-to-server FTP
• Update dependent services (Basecamp's FTP settings).
• Lock comments on the old server blog
• Clone the WordPress database (which I've already tested last month)
• Switch over the DNS name servers from Pair to FutureQuest, and wait for the new DNS information to propagate over the weekend.
• Cross my fingers

My email will also be up and down as the nameservers sort themselves out on Saturday and Sunday.

RANDOMLY THINKING...

Hm, that MediaTemple Grid-Server package is looking kinda tasty too.

Comment Spam Countermeasures

I've been noticing an increase in comment spam yet again. While Bad Behavior and Akismet are doing a good job of keeping the comment spam out of the blog, I'm also starting to hit some server performance issues. I'm not quite ready to move yet, so I thought I'd try an easy trick to see if I could alleviate server load due to heavy spamming activity.

Geeky notes follow.

Bad Behavior and Akismet Revisited

Someone asked me why I was using two spam plugins...didn't they do the same thing? So here's why:

Bad Behavior 2 is the first line of defense. From the Bad Behavior website:

Bad Behavior was designed and built by watching actual spambots which harvested email addresses, posted comment spam, and used fake referrers. By logging their entire HTTP requests and comparing them to HTTP requests of legitimate users, it is possible to detect most spambots.

In other words, it actually acts as a screening program to prevent suspicious software from "seeing" your website. It does this by doing a bit of behavioral profiling; software that does not play nice is not allowed any further. This means that the spambots can't even see the comment form, which means they can't leave spam. Legimate users with a web browser, though, behave nicely, so they are allowed to see the web page in most cases. Advanced users who have tricked out their Internet to be anonymous, for example, fall into the "spooky" category and probably get bounced.

Despite all the work that BB2 does, some spambots still get through. This is when Akismet comes into play. This plugin analyzes the comment text itself and determines whether it's spammy or not. Bad Behavior, by comparison, just looks at how the entire website is accessed, not the content. If you were to think of Bad Behavior as law enforcement, it's like an officer who arrests anyone that even looks like they might commit a crime. Akismet, however, waits for the act to be commited, then sees if it's criminal by consulting its database of known acts of spam; these are collected and analyzed by the network of Akismet plugin users.

So the combination of Bad Behavior 2, which vigorously beats off the majority of spam attempts, makes for a lighter workload on the server AND for Akismet, which does cleanup on a case-by-case basis. Since installing this combination of plugins several months ago, I've seen maybe 2 or 3 actual spam comments get to the moderation queue where I can give the final OK. A few legitimate comments have been caught, but even that has been pretty rare (maybe 5 or 6).

Never the less, I'm still experiencing some server sluggishness, so I thought maybe I'd tweak my setup.

Security through Obscurity

The phrase security through obscurity usually has negative connotations: any security system that relies on secrecy alone is pretty half-assed. You can't rely on secrets to keep things safe; you also need active countermeasures. It occured to me, though, that in this case security-through-obscurity thinking was appropriate.

Spambots aren't personal; they just blindly go out and try to access the comment posting mechanisms on blogs based on the default settings. For example, the file that does the actual posting of comments to WordPress is wp-comments.php which lives in a certain directory on every installation. A spambot knows this; it's just like an opportunistic thief trying the doorknob on your car or house to see if it's unlocked. It's just the obvious thing to try. It's easy enough to do quickly, and the payoff when it works is enough to justify the effort. It just takes one payoff.

Now, knowing thisimagine that you've hidden your doorknob. Or moved the door behind something. A thief who's in a hurry will just walk right on by your house...there are always easier targets. Spambots are similar; they need to make thousands of posts to be able to turn any kind of profit for their overlords, and there just isn't time to spend cracking your particular blog. There are lots of default installations of WordPress out there, and those are the ones that are being targeted. Spammers aren't out to make a personal connection; they need thousands and thousands of links to make their dark search engine optimization tricks work for dozens of sites. That means they will target high-yield scenarios like "default installations" of blogging software. They will also target sites that are protected by solutions like Akismet and Bad Behavior, if they've figured out a way to get around the countermeasures. Even in this case, however, they're not targeting your specific blog, but a particular plugin.

Filename Switcheroo

If you measure security by not allowing any accesses to your valuables, then security through obscurity is clearly not enough. A determined attacker will find your secret. However, since Spambots aren't determined attackers, and our goal is merely reduced violations, applying a little secret sauce is entirely appropriate.

I found Yami McMoot's write-up on renaming script files on the Wordpress Codex, and have changed my comment and trackback files to a random name. Spambots that try to access the default pages will get served a blank page; they'll just appear broken to the spambot, which will move on its merry way to the next target.

Now, this won't stop an attacker who's actually using a browser to access and enter comment spam, and I imagine that there are attempts right now to create an automated browser-based spambot that can automatically find and navigate to the comment box. Spambots that parse HTML would requires active countermeasures like Bad Behavior, or perhaps would have to use an entirely different comment posting mechanism (Flash, anyone?).

Summing Up

So I'm hopeful that this will reduce the amount of incoming spam by a bit more. The anti-spam fortifications in place now look like this:

1. Secret Entrance -- For automated spam scripts to access the comment posting mechanisms directly, they'll have to guess what random name I've chosen for these two files. The OLD files are there, but they're blank and will not invoke Bad Behavior. This procedural change is invisible to "regular" users.

2. Bad Behavior -- All visitors, normal people and spambot alike, are submitted to the screening. Spammy behavior gets you booted before you even reach the blog. Everyone else is allowed to see the website.

3. Akismet -- Any comments that are submitted will have been made by people (or spambots) that have made it through the two lines of defense. Akismet now looks at what's been written; spam is marked as such.

4. Moderation -- Any legitimate comment that's made it through steps 1 through 3 are held in moderation, until such time when I personally read and approve it. On approval, the comment is posted to the blog.

In my WordPress dashboard, Akismet shows me how much spam it's blocked since it was installed. I added a little hack to Bad Behavior 2 in bad-behavior-wordpress.php to display a similar message:

    function bb2_dashboard_stats() {
        bb2_insert_stats(true);
    }
    add_action('activity_box_end', 'bb2_dashboard_stats');

I'm not actually sure if adding countermeasure #1 will make a huge difference; I suspect my laggy server performance is just due to me starting to outgrow it. However, if there are a significant number of spambots just hammering the ser that bad behavior has to deal with, it might add up. Bad Behavior 2 reports about 150-200 blocked posting attempts a day, and Akismet blocks maybe 20-30 that get through.

We'll see what happens.

Moving WordPress Databases

Although I keep telling people I don't do servers, I'm looking into moving a friend's active Phorum-based BBS to a new host. I'm considering FutureQuest as the new host. I'd heard good things about them, though they're a little pricey compared to other plans.

This is a good opportunity for me also to evaluate new hosts for the blog. I'm currently using Pair Networks, who have been awesome for the past 8 years, but their database servers start pooping out when you get over a certain number of transactions per minute, and it's starting to become an issue.

This is the first time I've had to move a WordPress blog from one host to another, so I'm documenting the process. WARNING! GEEKY NOTES FOLLOW!

Different Server, Different MySQL Versions

Conceptually, it is as simple as uploading my current WordPress files to the new server, and then transferring the MySQL database that holds all my posts and stuff. POOF, it should work, right?

As it turns out, the FutureQuest DB servers are using MySQL version 4.0.x, and Pair is using 4.1.x, so I have to downgrade to an older server version.

MySQL Import Woes

There were three easy ways I found that would backup a Wordpress database, which I tried one-after-the-other.

• First up was Aaron Brazell's WordPress-to-WordPress Import exporter / importer. WordPress doesn't come with a WordPress importer, so that's what this package adds.

  The plugin exports your posts in an extended RSS format, which you can then upload via the import plugin from your browser. Great in theory! Unfortunately, my blog backup is 9MB, which exceeds the 2MB upload cap of this server for PHP. I may try hacking this to read the backup file from a location on disk, but I decided to try another approach.

• Next, I looked at Skippy's WP-DB-Backup plugin, which is included with WordPress 2.0, gives you the ability to produce a SQL dump file from the convenience of your admin panel, without all that fussing around with the command line.

  So I did this, and then tried importing the dump file into the new server using the standard mysql < dumpfile.sql style approach. It didn't work: 1064 - You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near 'DEFAULT CHARSET=latin1'

  Darn! This looked so promising.

• I moved on to Lester Chan's WP-DBManager 2.05 plugin, which allows you to do a lot more with your WP database. It uses mysqldump to create a compatible SQL dump file, which you can then use to re-initialize a new WordPress installation. SAME ERROR in attempting to import. DOUBLE DARN!

Downgrading from MySQL 4.1 to MySQL 4.0

After some digging, I found that MySQL 4.1.x added a different way of handling character sets, and this was reflected by the DEFAULT CHARSET command added to the end of the CREATE TABLE statements in the dump file. MySQL 4.0.x has no idea what this is, so it throws an error.

The workaround I found was to modify lines 32 and 36 of Lester's dbmanagerdatabase-backup.php file. I added:

--create-options --compatible=mysql40

...before the '|' character on line 32, and before the '<' character on line 36. The first parameter tells mysqldump to use MySQL-specific syntax for creating the table, and the second parameter makes those problematic DEFAULT CHARSET commands go away.

The Backup Procedure

So here's what I ended up doing:

1. On the CLONE blog: Install WordPress 2.0.4, plain vanilla, uncustomized. This is the "blank clone" on which my old blog will be imprinted.

2. Install Lester Chan's WP-DBManager 2.05 plugin on both old and clone blogs.

3. On my OLD server: I edited the plugin as described in the previous section, so the database backup file created would be compatible with the new server. Again, my current server uses MySQL 4.1, and the new server uses MySQL 4.0.

4. On my OLD server: Created a database backup with the patched version of Lester's plugin. Yay!

5. Then, copy the database backup to the CLONE server. It's in wp-contentbackup-db if you've successfully installed the DBManager plugin; just copy it using FTP or SCP to the same location.

Preparing for Restore

• First of all, my CLONE server has only a numeric IP address, not a domain name. I don't plan on transferring the davidseah.com domain over until I make sure everything works. So the URL I use for it is, for the same of example, is http://192.168.1.100.

• WordPress keeps track of your blog's URL in the database table, and uses that to generate links to various parts of your blog. When you do a database restore with DBManager, it overwrites your entire database, including that blog URL setting. When that happens, you won't be able to use the admin page, because you'll be redirected to your OLD blog everytime you try to login.

• To get around that, you need to be able to edit the database directly and reset the BlogURL setting. I installed PHPMyAdmin on the clone, to be ready to edit the database manually when I did the database restore.

1. To the CLONE server: Copy the database backup file to the wp-content/backup-db folder, where DBManager looks.

Now you're ready to restore the database.

The Restore Procedure

1. On the CLONE server: Copy all your plugins, template files, and so forth over to the new WordPress installation.

2. On the CLONE blog: Go to the admin page and click the DATABASE tab that DBManager has added. Go to the MANAGE BACKUP DB tab, and choose the database file to restore (you did copy it to the right place, right?).

3. Restore the database by clicking the RESTORE button! If everything goes well, you'll see a SUCCESS message. Congratulations! Your database has been cloned onto the new wordpress installation!

4. Visit the cloned blog server and see if the home page works. It's probably messed up in some ways if you've forgotten to transfer everything over, and you'll find that you keep getting re-directed back to the old blog if you click on anything. That'll be fixed next.

5. On the CLONE server: Run the MySQL Administration Tool of your choice. Using PHPMyAdmin, I went to the wp-options table and clicked BROWSE. I edit the siteurl option value to point to the CLONE server (not the CURRENT blog).

   Here's an example:

   My old site is davidseah.com. My new clone site is, say, on a local server 192.168.1.100, and WordPress is installed in a directory called wordpress. The new siteurl (at least for now) is http://192.168.1.100/wordpress, so that's what I should set the siteurl to. Your new webhost should have provided you a URL that you can use to FTP files to, so just use that.

6. You should now by able to login and visit the CLONE blog's admin page. Go to Options and change BLOG ADDRESS so it's also pointing to the new server for testing.

After all that, the cloned blog seems to be working, but I won't be able to tell what genetic defects may be lying dormant until they manifest. Database performance seems a little sluggish to me, so I'm not sure if the way that I performed this database move somehow messed up the DB. Maybe some MySQL guru out there can give me some pointers.

Next Steps

Before I move everything over, I need to make sure that my .htaccess is set up correctly, get ALL the downloadable files moved over, update my basecamp FTP settings, and also ensure that my new email addresses are set up on the new server. What a pain in the butt!

My DNS is handled by another registrar, so I should be able to just change the nameservers when I'm ready and see what happens. If it doesn't work out, I can just switch the nameservers back, and resume operations at my current host.

UPDATE:

• My Server Move Log, which also details the different way I moved the database with straight mysqldump and mysql restore commands from the shell.

Editable Comments and WP Cache 2.0

I've been making various improvements to my WordPress installation, hopefully making it a little easier to play here on Better Living through New Media. One long-standing peeve was the lack of user-editable comments. I had seen Andrew Sutherland's Edit Comments Plugin before, but had put off installing it because the post warned of the "complex modifications" needed to your theme files. As it turns out it actually isn't bad, but I had to fix a compatibility issue with WP Cache 2.0 to get it to work right on my install of WordPress 2.0.3.

Geeky notes follow.

The Installation

As I said, I found the mods actually aren't too bad if you understand PHP and WordPress at an intermediate level. There are basically only two steps:

1. Upload the plugin and activate it in WordPress' plugin administration window.

2. Edit your comments.php file. The variable is how your particular theme implements that file, so it is potentially confusing to those unfamiliar with WordPress internals. The readme.txt file that Andrew includes is very clear on what you need to edit. It's a delight to come across a technical document that establishes both "what you need to do" with "what it does" and "how it works". Excellent! There are about three things you need to change in the comments.php file, and it's actually pretty straightforward. I put this all off for nothing!

There was one wrinkle, of course, otherwise I wouldn't be writing this. WP Cache 2.0, the cache plugin I use to lighten the load on my shared host, caches the comment so your edits don't show up after you're done. They are saved, but you can't see them until the cache expires or you edit again. Annoying.

The Modifications

First, I went to my WP Cache Options in the WordPress Admin panel, and added jal_edit_comments to the REJECTED URIS list. Save the strings. This prevents any page that uses the Edit Comment Plugin's method for communicating with WordPress (a GET value called jal_edit_comments) from being cached. It occurs only when someone clicks the "edit" link for an editable comment.

Second, I modified my copy of jal-edit-comments.php to detect WP-Cache 2.0 and delete the associated cache file for the current post being commented on. As far as I can tell it works. I added the following function:

// if wpcache is installed, delete cache file for this page
// .. added by david seah 07/14/06 
// .. called by jal_edit_comment_init(), jal_do_edit()

function jal_purge_wpcache() {

  if (function_exists('wp_cache_get_cookies_values')) {
    // import wp-cache configuration
    if (@include(ABSPATH . 'wp-content/wp-cache-config.php') ) {
      // strip out everything after the ? in the uri
      $uri = preg_replace('/?.*$/', '',$_SERVER['REQUEST_URI']);
      // create wp-cache 2.0.x compatible filename
      $key = md5( $uri . wp_cache_get_cookies_values() );
      $cache_filename = $file_prefix . $key;
      // delete the cached file
      if (function_exists('wp_cache_clean_cache')) {
        wp_cache_clean_cache($cache_filename);
      }
    }
  }

}

Now you have to modify two other functions in jal-edit-comments.php:

• Modify jal_edit_comment_init() by adding the following right after the first echo statement:

  echo '<p><input type="hidden" name=...
  // dseah: purge wp-cache file
  jal_purge_wpcache(); 
  

• Modify jal_do_edit() by adding the cache-purge call before the first die() statement:

  // dseah: purge wp-cache file
  jal_purge_wpcache();
  die();
  

• Now purge all the cached files using WP Cache's option panel, and you should be set.

What's Supposed to Happen

WP Cache 2.0 creates a unique filename for every web page it caches, and stores that in a special cache folder. The modifications I made to EditComments recreates that unique filename, and passes it to WP Cache's "Delete Cache File" function at two critical moments:

• Clicking the "edit" link adds a special GET variable called jal_edit_comments. This is intercepted by jal_edit_comment_init(), which now also calls our new function to purge the cached file. This makes sure that if there's a timeout, the cached page is nuked, otherwise the "edit" link will still be served up.

• Clicking the "submit" button invokes jal_do_edit (), which steals the comment text away from the usual comment handler (wp-comments-post.php), handling the database insertion itself. After that's done, it redirects the browser back to the comment page before terminating the script; however, before we do that, we purge the cached page again, because we want to show the updated text (the edited comment) instead of the cached version (which has the pre-edited comment)

So that's what I think I'm doing, anyway. It appears to work on my installation of WordPress 2.03, WP Cache 2.0.17, and Edit Comments 0.3 Beta. There may be a situation where some of the "You aren't allowed to edit this comment" messages won't show up if multiple people are accessing the same page and therefore thrashing the cache out from under each other, but we'll see what happens.

Availability

I thought about making my modified version available, but I'll just let the author of the original plugin know after I do a bit more testing. I don't want to create a technical support headache with two versions of the same plugin, with him having to support my possibly-buggy code. And if you figured out how to install WP Cache and Editable Comments in the first place, you probably can apply these modifications yourself. Good luck!

Flickr Integration

In celebration of the Independence Day, I upgraded my Flickr account. Flickr has really been impressing me lately, so for $24/yr, I was happy to do the upgrade to Pro so I could have more albums. I figured it might be nice to actually show a few more photos on the website too; my blog is rather, er, word-heavy, so providing an alternative means to browse content seems like the thing to do.

I am using FAlbum for the Flickr integration. It's pretty cool! Geeky notes follow.

Installing FAlbum

FAlbum is a WordPress plugin that talks to Flickr through the Flickr API. It's a pretty nifty plugin, though I was almost convinced it wasn't what I needed. A few tricks to know:

• I had some hiccups with the installation; this post at idano.com cleared up some of the requirements for the .htaccess file and page creation. After that, it was a matter of chasing down the little theme incompatibilities.

• If you still have a WordPress 1.2-style theme all in a single index.php file (as this site was until 2 hours ago), you'll get a few errors and a mysterious lac of formatting. You'll need to modify the wp-content/plugins/falbum/wp/album.php file to eliminate the get_header(), get_sidebar() and get_footer() calls. This file is what generates the actual page layout on your photo URL. I decided to break out my header, sidebar, and footer into separate files like a good boy.

• I had to dig through the FAlbum_WP.class.php file to figure out what the options are, as the documentation is still pretty light. You'll also want to familiarize yourself with the content of the ../falbum/styles/ directory; in particular the falbum.css.php file so you can customize the layout for your theme.

• I am having one cosmetic issue with the default size that Flickr returns. It's 500 pixels wide, and my default column with is 458 pixels; as a result, there's some nasty resizing artifacts in the browser. Very irking.

One thing I'm concerned about is the additional processing load FAlbum may impose. For example, there's a giant XPath library class included with the plugin that's over a megabyte in size...that can't be good. I'm also noticing that the cacheing is not really working correctly; will have to track that down :/

That said, it is pretty neat that people can browse my Flickr photos without leaving the site. For one thing, I can add a real portfolio section again!

Social Bookmarking Twiddling

:http://del.icio.us I was chatting with Brad over the weekend about making it easier for people to find our content, and he mentioned a plugin that would automatically add links to [del.icio.us], digg, and other social bookmarking services. The idea is that by making these bookmarking icons more accessible, people are more likely to bookmark interesting posts, which theoretically drives traffic.

Geeky notes follow!

Notable

I hunted down Cal Evan's WordPress nifty plugin Notable and installed it. Couldn't be easier! Every post now has 5 service icons; Notable actually supports a lot more, but I thought I'd start with the five ones that I had actually heard of first, and add more later if it didn't start getting out-of-hand.

Because I can't leave things alone, I did make some tweaks:

• The graphic icons were of varying heights, which made them float unevenly in my layout. I made them all the same height (now 16 pixels, faded 50% to better work with my layout).

• I added text to identify the service each icon represented. Rolling over the icons with the mouse does reveal which service the link goes to, but I am not a fan of rollover information. Help is one thing; information should be laid out where you can see it. Personal preference. Plus it looks kind of cool.

• I fixed some typos in the function notable_define_array() that caused the output to fail XHTML validation.

This last case was the common "& where an & should be" HREF encoding problem, but I couldn't find it. I eventually figured out that that the digg, magnolia, and segnalo post_url values did not have their &s urlencoded properly, though all of the other ones were.

What made this typo particularly tricky to fix was that I thought I could just fix the definitions in notable_define_array(), not realizing that these values are copied once upon first-time installation of the Notable plugin. After that, they are pulled from the database and never updated even if you fix the typos. I was tearing my hair out trying to figure out if it was the browser cache, some mysterious PHP cache, or maybe a plugin post-processing my ampersands. Nope.

The workaround is to put the following code just before the blogbling_post_options (notable, $notable_settings) line at the end of the function:

    // ds: force update of post_url values
    $work = notable_define_array(); 
    foreach ($notable_settings['sites'] as $site=>$values_array) {
        $notable_settings['sites'][$site]['post_url'] = $work[$site]['post_url'];
    }

After doing that, the XHTML validates again. Whew!

Another minor gotchya is the "image path". I originally thought this was the full path to the plugin's image directory, but it's actually part of the URL that Notable uses to generate the <img> tags to display the icons. In other words, you're filling out this:

http://www.yoursite.com/notable_image_path

not this:

/home/virtual/yoursite-com/htdocs/wp-content/plugins/notable_image_path

Anyway, now that I have these little icons in place, I'm finding their colors too distracting, but maybe that will encourage people to click them. I'm a little doubtful that this will make a difference, but we'll see.

UPDATE: I couldn't stand it so I faded them out a bit.

The experiment in distributing original content continues!

Bad Behavior 2 Beta 1

Some nice commenters let me know that they were still having trouble posting comments and receiving errors. This was due to the anti-comment spam plugin I was using, Bad Behavior 2. Compounding the issue was the cryptic error messages BB2 generated when its "5-second wait-before-you-can-post" trigger was activated. I eventually modified my copy so it didn't give the people the impression my entire site was on the verge of collapse; most people just saw ERROR and tons of tiny text, and (naturally) assumed that some kind of severe system error had occured due to errors in my PHP coding or my ISP's server configuration. That is in fact what I thought until I read the text more carefully.

Anyway, today I was hapy to see that Beta 1 is available. It gets rid of the 5-second wait that caused the problems I just mentioned.

• Just installed it. Crossing my fingers.

• I was about to give WP-Hashcash a try again, but people have been reporting that it's also letting spam comments through again...figured I would try BB2 first.

• I also deactivated Spam Karma 2 and enabled Akismet 1.15 to see what happens. I know, it looks like bad debugging methodology...I should just keep BB2 on for a while first to establish any change in behavior, then try swapping Akismet in. However, the input is pretty variable: new spamming techniques arise every day, as do new spammers and spam surges. I wouldn't be able to tell, without spending a lot of time I don't have, what the causal relationship is between spam stopped and a particular plugin combination. So I might as well just see how they work from an administrative perspective.

Without these plugins, I would have to shut off comments. It is incredibly irritating that these spammers STEAL OTHER PEOPLES TIME AND PROPERTY to make their money. This starts to make the non-anonymous Internet look attractive, but then you have the problem of TRUSTING THE GATEKEEPERS of the system to maintain the egalitarian policies of a Free Internet in the face of commercial greed. Sigh.