Oblique Searching Strategies

I'm surfing instead of working, so to win back some productivity, I'll describe my basic searching strategy!

When Google doesn't find what you want, it's because you are probably using an overly-general search query, or one that has been polluted by e-commerce junk information. Argh!

To get to the real opinions on the Internet, you need to search for words that people use, as opposed to the lifeless copywriting you see in a lot of "professional business communication."

• Look for reviews and opinions. For example, I've been looking for information on a particular computer peripheral, something called the DAC-100. So I do a search on DAC-100 review, DAC-100 opinion, and even DAC-100 compared. I'm using the words you're likely to see in a review, as opposed to marketing copy.

• UN-search for marketing copy. I remember looking for reviews on a particular ricecooker (the Zojirushi NS-KCC05), and the entire search result space was saturated with Amazon and Amazon.com feeder microstores: basically, the exact same information over and over again. Use this against them! Find a unique phrase in the copy and ignore it in the research results by adding a - in front of it. For example, if the ad copy says something like, "re-heats rice to the perfect serving temperature", do this search: zojirushi NS-KCC05 -"re-heats rice to the perfect serving temperature". You'll not see any results that contain that phrase. Thank God.

• Try Model Numbers or other ID. This sometimes helps to isolate a particular product. Use Amazon or Epinions to narrow down which model number you're interested in. However, as models are often quickly replaced, you might not have much luck finding the dirt on a specific generation of unit. Sometimes Shelf Keeping Units (SKUs) are helpful--those are the numbers on the barcodes you see on retail packaging, and eCommerce sites sometimes list them in their online catalogs.

• The more specific you can make your search terms, the better your search results will be. There are also nifty advanced search options at Google that can filter your results by date range, file format, and so on.

When all that fails, it's time to go to the mattresses. Try other information sources on the Web to learn more about the topic, and search on the cues that point to expert commentary.

• Search eBay. When you can't find information or photos on some object, you might actually find it on eBay, which is not indexed by Google. I spent days looking for a reference photo of a Hebrew keyboard, eventually cobbling one together out of scraps of information from programming reference articles. Just yesterday I searched eBay and a dozen photos of the keyboards were there. Sometimes the sellers write a great deal of personal history about the items in general. Don't forget to search past auctions, and it may be worth looking at other specialized auction sites (gunbroker.com, for example) that cater to specialty markets. In either case, you might find a seller who is willing to tell you want you want to know about the item.

• Search Wikipedia. Wikipedia entries don't often get ranked high in Google results, so go there first. Chances are, someone's written something about what you're looking for, and some high-quality links have been already researched for you.

• Search The Top Specialty Information Sites. For nerd stuff, that means going to Slashdot. For digital photography, that's Digital Photo Review. And so on. You'll probably start to get a sense who's on top once you've visited a dozen sites, particularly smaller ones that have lists of links to other informational sites. They are usually the same, so eventually you will see who they all point to.

• Search USENET. USENET was the primary Internet discussion system, a kind of global bulletin board system, until the World Wide Web balkanized the infosphere into millions of website-states. Prior to the mid 1990s, USENET was our primary community-driven information database covering thousands of topical interests. It still lives on at Google Groups. USENET will often deliver pieces of insight you will not have found on the Web.

• Search The Blogosphere. Technorati, for example. [UPDATE] Google has added blog searching. Try that!

• Search FAQs. One byproduct of the USENET era was the creation of Frequently Asked Questions (FAQ) post. Groups like comp.graphics were often inundated by newcomers to the group who asked the same questions over and over again. Over time, the FAQ postings have become quite comprehensive. While not all of them are actively maintained, they represent a trove of practical information on a broad range of topics. A lot of them are archived at faqs.org. You can do a full-text search.

• Search Magazine Indexes. Sites like FindArticles.com index articles in magazines that may not always float to the top of a Google search. Worth a try.

• Search Amazon. Type in your search subject, and see what books pop up. The titles alone, the people mentioned, and the commentary left by other amazon shoppers are immensely useful when mapping out a new information space. You can then use that information to narrow down your search on Google. Searching the Wishlists for a particular book title can also help narrow down the search field.

• Find Secondary Information Sites. There is often useful commentary on the smaller personal websites. They don't rank high on Google, so use a blog indexer like Technorati and see who's linking to that major blog you like. Use Google to find sites that link to two or more other good sites you've found. Are there people out there who read both Dooce and Crossroad Dispatches, and feel strongly enough to provide a link? That might be an interesting person to read.

• Look for Color Commentary. Sometimes you really just want to get some random guy's opinion. I like to search for terms like "really sucks", "I think", "I would", "technically speaking", and other phrases that someone giving your their opinion might say. The general principle here is to mix the topic of your search with some kind of "biased phrase." Slang that's in use by a particular generation can be useful too. For example, "friends with benefits" versus "casual sex" versus "free love"...all used by different generations (20-somethings, 30-somethings, hippies...) Likewise, if you are looking for negative opinions, try words like "stupid", "dumb" and "sucks". Use your imagination, and you'll be rewarded.

• Look for Expert Cues. The relative obscurity of a word can help narrow down the results. So instead of searching for best plasma tv television, which will turn up crappy e-commerce websites, look for color balance reproduction fidelity plasma tv television...you're more likely to get an article that's written by someone who edits Audiophile magazine on the side. Or if you're looking for something more academic, use academic words or titles. "Ph.D.", "professor", "research", and "bibliography" might slant the results more in that direction.

• Hunt by lineage, not by category. Sometimes, keywords fail to reveal useful search results because the keyword space is heavily saturated by another meaning or by common-place daily use. If you're looking for the rare and the exceptional, you need to search for topical anchors: key people, key philosophies and key innovations that shaped the field you're searching. You first may have to read a general history of the topic you're searching to get an idea of what those key anchors are. People who are learned in their field will often reference key principles as a statement of their belief; following these chains of beliefs and the sites that they link to will help you form an alternative search topology that Google is unable to provide through keyword techniques.

• Search Patent Databases. When you're looking for a utilitarian or specialty item that isn't directly used by the average consumer, the trail often goes cold, or is polluted by unrelated commercial offerings. Find out who holds key patents by searching the US Patent Office Database. There are other patent registries online too, so just look around.

  The wealth of information you can gain from patent applications can put you back on track: names of inventors, manufacturers, and related patents--not to mention the description of the apparatus itself--can give you more to Google. Or look for the patent number on an actual object you're interested in (if there is one) and look it up online. That can save you buckets of time.

• Don't Neglect Your Local Research Librarian! If you are lucky enough to live near a university library, or even a decent public library, go check it out. The good libraries have people trained in navigating piles of information...librarians! The specialist ones know where everything is. You can also find out how to access the library's card catalog over the Internet, which might save you some legwork. In a pinch you might go to Barnes and Noble or Borders or a decent independent bookstore, and ask someone where the books on XYZ are.

• Look for Accessibility / Required Policy Markers. I just read someone complaining about trying to find the "real" website for a hotel; there are so many two-bit hotel portal sites that they drown out the real sources. I noticed that most of the hotels include the phrase En Espanol, whereas the portal sites tend not to worry about that kind of accessibility because it's expensive to implement for relatively little gain. Likewise, including a phrase like Section 508 (a requirement for government sites) might narrow down search results. Try slogans associated with the company, if you know them...a portal site is unlikely to completely parrot the corporate line, but you can bet that the corporate web team had that requirement tattooed on their butts before they started the project :-)

• Search Flickr. Those photo sharing websites, with their full keywording and text descriptions, provide a secondary image databank in addition to stock photo and Google Images. I've found some pictures of pretty obscure stuff much more quickly on Flickr than I have through Google. Try it!

• Search 9rules. 9rules is a blog network of several hundred sites that emphasize quality of conent. They've recently added 9rules search, which is a quick way of seeing what some thoughtful writers have said about a topic of interest to you. It also searches through a new other services like YouTube, so this is an interesting alternative search you can try. Full disclosure: This site is a member of the 9rules Network.

• Search Social Bookmarking Sites. Sites like Del.icio.us are filled with links to the web content that people find useful; they are therefore much more likely to be interesting than general search, and the collections of links may reflect a theme or approach to the topic that you hadn't considered. And even better: you may actually discover someone who has relevant interests that you can contact directly...you never know!

Of course, all these techniques can be mixed together. This will give you raw data to work with, from objective and subjective sources. Determining whats's useful and what's not is up to you. Have fun!

Get Published on Amazon

Kevin Kelly posted an article on how to get published on Amazon. It doesn't take a whole lot of cash outlay (around 150 bucks plus two copies of your work), though you get only 45% of the listed price. Kelly notes that this isn't a way of making money, but to get your message out there by leveraging Amazon's huge traffic and e-commerce system. Cheaper than the vanity press, with more eyeballs! Via BoingBoing.

Ad Hoc Knowledge Systems

I caught an episode of NPR's Day to Day regarding the creation of expertise-sharing websites at the company commander level. At companycommand.com, military commanders at the company level (a company being from 100-200 troops) exchange practical tips on how to get the job done. Say you've got to lead a convoy out of Falujah...ask your fellow commanders on this password-protected site for their advice. Listen to the Day To Day audio segment.

While the companycommand.com website is not open to the general public, there's a link to Tomoye, the company that apparently makes the knowledge system behind it. It seems to be a social network package designed to facilitate the creation and management of what they call a Community of Practice. The powerpoint demo provides a good general overview of what the product is supposed to do, though I wonder how many people really get it.

You'd need motivated leaders and contributing experts to make any community happen, but it's interesting to see how social networking is developing from beyond simple roll-ups like LinkedIn to make it easier to happen. A few months ago, Meetup added something similar, but that site seems pretty dead these days.

Write-Fu

Wikis have been on my mind lately, tangentially through some posts on the Markdown mailing list regarding the addition of tables, images, and so forth to its syntax.
I came across this posting regarding productivity tips via Weblog Without Honor. The suggestion that a personal wiki or Wiki-like editor to help organize on-the-fly is very compelling.

Think of it like this: Markdown makes it easier to seamlessly write and format as you go, without a lot of clicking around with the mouse. It's even better than most keyboard shortcuts, because you don't have to grapple with the control and alt keys in awkward ways.

Awkwardness kills productivity, and it pervades modern software. Don't get me wrong, keyboard shortcuts are great, but there are very few software application that have keyboard/mouse interfaces that really feel seamless. In fact, there are only two pieces of software that I've used that really achieved this: * The pixel painting program DeluxePaint, published by Electronic Arts * The word processor WordStar from MicroPro International

Both these applications had a unity between keyboard/mouse interface and the task at hand. The shortcuts chosen and philosophy behind tool mix allowed you to keep your hands working over the home keys and mouse. Because of this, you could keep your eyes focused on the exact space that you're working. It seems like a small thing, but you drop into "the zone" faster without the hundreds of tiny interruptions caused by looking for a button, shifting your hands to reach the control key, and so forth.

You might argue, "Yeah, but you eventually learn the shortcuts it's not that much time lost really. Just look at the screen!" You're right. But you're missing the point. I don't want things to merely work...I want to achieve work-fu... the fluid execution of creativity and productivity. Every glance that takes you from the task is a wasteful glance; it is better to excise it from your habits. Every extra inch of movement is a waste of energy and time. Excise these wastes from your habit, and divert that energy into production action!

There's a similar concept in military aircraft called HOTAS -- Hands on Throttle and Stick -- that puts essential task-oriented functions right on the primary flight controls. You know, things like arming weapons, selecting targets, and so forth. You could mount those controls in a very logical and easy-to-read panel on the dash, but the time it takes to take your HAND OFF THE CONTROLS to manipulate them forces you to not control the aircraft fully for that time. Of course, you end up with a potentially confusing cluster of buttons and knobs on your controls, but they've been shaped by years of experience and (hopefully) do not alter their primary function of controlling flight.

So back to the task at hand: What is my Primary Task? It's thinking and writing. What's the problem? Structuring and connecting more than a few pieces of information because more of an exercise in finding things than actually structuring them. Authoring a good hypertext document takes a lot of cutting and pasting of different kinds of information all over the system. My ideal information management system would allow me to write, organize, and link text with other media by keeping my hands on the keyboard 90% of the time.

TiddlyWiki

I stumbled across vroop.com while reading a little about daring fireball, which is the website for John Gruber (the creator of Markdown). Two people who care about the way software works in an intuitive, use-aesthetic sense. On Vroop I read mention of TiddlyWiki, a client-side "Personal Web Notebook" based on the whole Wiki shared document concept.

TiddlyWiki looks like it might be the non-linear thought processing application that I've been looking for. I often write stream of consciousness style, defining terms and grouping things as I go along. Usually this is a somewhat tedious task, requiring many passes to refine the core ideas. But the organic and self-organizing properties of Wiki, with the ability to automatically define entities by mashing WordsTogetherLikeThis, could provide me a way of writing and organizing on-the-fly.

My Favorite Software, Part I

LAST UPDATED: September 26, 2004

I'm picky about the tools I have installed on my computer. Every once in a while, I flip out and look to see if there's something better out there than what I'm using. I'm fairly happy with what I'm using now, so here's what I'm using right now on my Windows XP box.

TEXT EDITOR

My text editor of choice is still TextPad, currently version 4.3 on my boxen. It's loaded with all the features that a 2000-era text editor would have: syntax highlight, multi-document management, and regular expression searching. It's also got a useful binary view mode, which is useful when figuring out file types quickly (I just append ".bin" at the end of the filename). It's still missing a few things that I'd like: function recognition, code folding, and maybe FTP connectivity. NotePadPro is perhaps more capable, and SciTE looks like a possible replacement. But TextPad doesn't overload its toolbar with giant ugly buttons, and it loads and feels faster than just about any other full-featured text editor I've tried.

EMAIL

I use TheBat! 3.0. I've been using it since around 2001, when I first became incensed at total unimpressiveness of Eudora 4.x. Outlook and Outlook Express, with their dodgy record on reliability with large mailboxes and tendency to lock you in via broken data exchange, not to mention the continuing security problems...not even a consideration. I tried PegasusMail and some others that escape my memory, and it was TheBat that I stuck with. It's fast, it has great backup and restore, and once configured to do away with its "stock ticker" thingy, it's lean and mean. It also will show you meta information, such as full headers and email parts, much more readily than just about any email program I've used. And it takes pains to make itself less susceptable to evil executable attachments and HTML (it actually has its own HTML rendering engine, instead of using the Internet Explorer engine which seems to have a new exploit every week). As a result, the program isn't as pretty compared to other mail clients, but it's cleanly designed.

ZIP UTILITIES

The earliest version of Stuffit for Windows was a simple shell extension. You double-clicked a file to expand it, which it would in same directory as the archive. To compress files, you selected them and right-clicked to choose "Stuff" or "Stuff with Options". When it added ZIP compression, it became my current favorite archiving tool. I still use it in the way I describe. The current version I'm using, version 8.0, adds a lot of useless stuff like an archive browser with huge buttons and marginal additional utility. But it beats the shit out of WinZip and its wizard-driven ilk, and it doesn't created corrupted archives or extra mystery folders as its early progeny did. Archiving with Stuffit, once you turn off all the new stuff, is fast and seamless. I'd just get the regular version...I don't use any of the other features.

But what about corrupt archives? For years, I thought I had lost an important archive from 1996. But one day, having discovered a separate archive was also bad, I started to doubt the integrity of the unzipping software. Although the name of the archive format, "ZIP", hasn't changed, there have been internal changes. there was a chance, I thought, that modern Zip programs had forgotten how to unzip the old standards. There's one last-ditch program called Info-Zip that might serve your needs, and recover partial archives. Apparently it was written by archive maintainers, who have to deal with ancient 80s-era zip files. So compatibility is very important to them, as you can imagine.

DATA SYNCHRONIZATION

SynchronEx will let you compare the contents of two folders, and synchronize them so they are the same. The registered version can do one-way syncs (for backup) . Unlike most synchronizing programs, you can get the control you want by editing a script with a regular text editor. You execute the script by double-clicking or right clicking to do the sync. It will by default just pretend to run the sync to tell you what it would do...if that works, then you can choose to actually execute. The nice thing about SynchronEx over others is its transparency of operation and nearly-seamless integration into Windows Explorer. There's no need to hunt around for a program. Just right-click and select SYNC and it goes. It will work across the network just fine too, so I use it for production backups to my file server. The first time you install it, you'll wonder if it actually did anything, but read the information that pops up on the screen. While it's initially kind of confusing, it's well worth the effort to learn.

ONLINE TOOLS

• QuickEscape quickly escapes HTML text for when I need it.
• This version of XMLTidy comes in handy when I need a quick XML reformat and the data isn't sensitive.