Mission Improbable Gospel Music 2: Chiseling out a Chord Progression

As I wrote a few days ago, I had decided that I would try to write a Gospel song despite my inability to credibly play an instrument and lack of experience with music composition. The reason: it seemed like an interesting challenge at the time, and I was quite ready to be a clueless novice in the process.

Despite my inability to perform, I do have a history of having been exposed to music environmentally, and it seems to have wedged certain patterns into the part of my brain. Yesterday's comments were heartening, as some of the decisions I made about chord progression--acquired by mashing keys on the digital piano until they sounded right--were on the mark. Woot!

I spent a few hours over the past couple of days figuring out how to actually get from my scribbled notes into something I could actually share. Geeky notes follow :-)

What I Have Noticed about Music Structure

I've noticed that the popular music I listen to has a definite pattern to the combination of notes that are played in each temporal group. What I'm calling "pattern to the combination of notes" corresponds, I think, to the chords selected from the major or minor scale in a particular "temporal group"; this group is called a bar or measure. Part of the joy of music seems to come from the ((combination of predictability of the pattern balanced with surprise**; a lot of popular songs follow a formula of some kind where there are certain number of bars with certain combinations of chords, but the way the notes are expressed is quite infinitely varied.

There are 12 notes in western music, and all of our songs are made of those combinations. From a graphic designer's perspective, one could think of them as the available named colors. As with colors, certain notes just seem to go well together, while others don't. I once tried to figure out why this was by looking at the relationships between pure sinusoidal tones, as I know from Electrical Engineering that sine waves are the fundamental building blocks of all more complex waveforms. As it turns out, the waves that sound good together tend to share harmonics in interesting whole-numbered multiples, but I don't know if this really what creates the impression of harmony.

I came up with a progression of chords (each of which consists of 3 notes) that sounded "right" to me. I could have started with any good-sounding chord, but I found that for the next chord to sound right, only certain notes were available otherwise it would sound weird. The pattern I was aware of reminds me a lot of a good episode of The A-Team, the formulaic-yet-enjoyable 80s television hit featuring Mr. T:

1. 4 bars of introduction, which sets the style of the song. Like when the A-Team gets contacted by the help-seeker of the week.
2. 4 more bars of "character building", fleshing out where this song is going to go emotionally. Very similar to Hannibal working out how to handle the problem with Face, Murdock, and B.A.
3. 4 more bars of "conflict setup", which feels like an intermediate mystery to be solved. The Team is dealing with the operational challenges of the mission, but before the final conflict has arisen.
4. 4 more bards of "conflict resolution", a plot point that is closed (for now). The Team is finally at the place where they need to take action to prepare for the final act.
5. 16 more bars of "whooping it up", which is when the show features the "Let's build something kickass and shoot up the place" montage.
6. Repeat as necessary.

I found I couldn't break certain expectations outside of certain intervals, or it sounded wrong. I couldn't change a chord in the middle of a bar, because...well, I didn't like it. I couldn't change the key until at least 16 bars of that setup had occurred...key changes are like changing the lighting in a room in a movie. And even then, it seems like the keys need to share at least some notes immediately before the break so there is some commonality. The chords within a series of bars had to relate to each other in some way too, in some manner I can't quite grasp. It might be pure familiarity at work here, hundreds of years of the same structures informing thousands and thousands of songs. Maybe some of them just were easier to play on certain instruments. Others just are good sounding; not all chords are created equally when it comes to harmonic bliss. My friend Lee pointed me toward equal temperament as a concept; it turns out that all these various "official notes" western music are a hack so a piano can have a fairly decent go at playing different scales "equally well". Which really means "equally bad"...they're close, but not exact.

I was dimly aware that chords and chord progressions describe a song pretty well; it's like using a grid in graphic design to pre-solve certain spatial relationships and create proportional harmony in the division of space. If I could just pick a chord progression that sounded OK to me, that would solve a lot of problems automatically. This presumes that I could actually tell if the notes I was picking were in the progression or not. This is something I seem to be able to do, but I sometimes wonder if all musicians actually can tell. I have lost count of the number of college band guitar solos that seem to be connected to a different song than the one they were playing.

Picking a Progression of Chords

And then...I was stuck. I couldn't pick the progression because I couldn't hear the rhythm in my head to further constrain the problem sense. I tried humming a few things to myself, trying to match the keys on the keyboard, but this felt like spinning my wheels. I eventually realized that I could impose some additional structure by creating a fake lyric that just had the right number of syllables in it. I basically picked cliche phrases from half-remembered gospel songs and piled 'em up to create a few verses, then hummed how I thought a singer might deliver them.

To find the right keys, the technique I used was to just play all the keys until I found the one I wanted. The process was very similar to sketching very loosely with lots of overlapping lines to outline rough shapes: our eyes pick the line we want, and then we ink it. In a similar way I thrashed keys that were in the direction I wanted the song to go, and ignored any note that didn't sound right. Once I identified something that sounded good, I wrote down the name of the note based on the one thing I remember from piano lessons: how to find C. I scribbled these notes down on my paper, and used them to play the chords over and over again while humming my fake lyrics. This was a laborious process because I had trouble matching my chord notes to the actual keys on the keyboard, and my dexterity was poor. However, I did manage to come up with this chord progression in the key of C major for the first 8 bars:

GCE - BEG - ACE - GBD  
ACE - CFA - GBF - GCE

Each group of notes is one measure, and they are all played simultaneously to "fill the space". When I played it out, using an organ sound, it sounded "right" but it was also very filling. These are the big meaty-sounding chords, with lots of harmonic shared relationships. I believe Lee called these "major triads", but I'm not sure if that's what I ended up with. It did sound familiar and somewhat churchy, though as I mentioned in yesterday's comments that that last group (GCE) seemed to "end" the song rather abruptly, leaving me "no where to go". I was fascinated to learn from a commenter that this phenomenon actually has a name: cadence, which is the "punctuation" of music. Some progressions just sound more "final" than others. Neat.

Note Transcription

While I had some chords down, I wasn't able to really experience them. I can't play the keyboard well enough to get a real sense of the song. Fortunately, I have a tendency to buy music gear whenever I think I might actually have the drive to learn something musical "for real". I also have some digital sound editing software that I use for audio storyboarding, editing, and digital media asset production. What I needed to do was enter the notes into a music sequencer software package, then render the sound to an MP3 file.

My MacBook Pro is running Windows XP natively, and into this is plugged an M-Audio Axiom 25 USB Keyboard Controller that I picked up 2 years ago with the idea that I might actually use it to learn how to use Reason, one of the first really cool virtual studio products that not only looked pretty, but actually didn't crash every few minutes. I unpacked the Axiom for the first time about 3 hours ago, and I'm happy to see that it actually works. :-)

I happpened to already have Sony Acid Pro 6.0, a multi-track music creation and sound synthesis package that I use for creating audio soundtracks from multiple sources, though it's been years since I've had to do this kind of work. This is the companion software to Sound Forge, the sound editing package that I've been using for quite some time for tweaking audio at the sample level. You can think of Sound Forge as the audio equivalent to Photoshop because it creates assets, while Acid is more akin to something like InDesign because it combines and layers assets you've created elsewhere.

Acid Pro 6 has a lot of music synthesis stuff built into it, and after some fussing with it I got it to recognize the Axiom 25 and input notes directly. Then I discovered I could actually draw the notes directly in with a pencil tool.

After entering my chord progression, I heard it played back to me for the first time at a measured pace, and realized I didn't like the second group. It sounded kind of awkward, so I tweaked it in the program until it sounded more interesting and less like a "full-stop". Commenter Steve had written to suggest another approach in his earlier comment, but I didn't refer to it because I wanted to see what I would come up with. You download the zip archive of GospelTest01 and listen to 0410-GospelTest00.mp3 to hear the progression, or scroll to the end of this post and click the audio player button to hear GospelTest00 and GospelTest01.

Churchiness and Sonic Space

As I listened to my chord progression, I felt a bit of despair: it sounded very boring and, well, predictable and lame. It reminded me of a graphic design faux-pas I see a lot from non-designers: the gradient fill. What happens is that the non-designer sees a great big white space, and they are overcome by the urge to "put something pretty" there. Rather than compose something tasteful, the gradient fill comes to the rescue to add some "style". It almost always looks terrible unless applied with some subtlety. I'd just done the same thing, musically. Oh dear.

Then I remembered: the chord progression just provides structure so other elements can play on top of it. Since I had the chords entered into Acid, I told it to loop continuously while I noodled around on the keyboard and tried to imagine someone singing on top of it. The chords then became less dominant, and suddenly the flatness went away. It opened up. The name of this file is GospelTest01.mp3 if you want to hear the difference.

A few notes:

• I did take the "gradient fill" metaphor farther in this file by using the 80s pop music equivalent: the synthesizer string fill to add moodiness and depth. It actually is starting to sound like something, despite my shameful use of synthesizer cheese. At least I am not adding orchestra stabs to "punch it up" or gating my drums.

• There's an interesting thing that I noticed in the 2nd-to-last chord I chose, which is its disharmonious quality relative to the other chords. It also sounds like there is a hole in the middle of it, which creates a slight anxious feeling. The last chord somehow "seals the hole" and eliminates that anxiety. I think I've heard this before in other hymns.

• Because I'm entering in the notes by hand, there is no live performance feel to the vocal part. It all sounds very robotic, because the notes are "quantized" to strict note boundaries. I applied Acid's Groove tool, which attempts to introduce some liveliness by slightly offsetting the timing. I think it did something, but my sensitivity to this kind of note spacing is fairly poor. It reminds me of kerning, which is the art of spacing letterforms so they look "even". Some people have the knack for it, able to see subtleties that I can't detect. And so it might be with musical timing. I left everything quantized here as a guide; I imagine a singer would know what to do to make this sound much cooler..it sounds pretty broken and clunky to me right now, especially listening to it "cold" without my imagination filling in the blanks.

Next Steps: Lyrics and Emotional Progression

As I was laying down the vocal part in 0410-GospelTest01.mp3, I had certain aspects of the yet-to-be-written lyric in mind:

• The singer starts by lamenting the difficulty of living
• The singer starts to realize that he's already half saved, he just needs to let something go
• The singer bursts into joyful celebration of salvation of some kind, hallelujah

The first 8 bars I have are that first part: lamentation. It's heavy-sounding and doesn't kind lift off except for one single note that rises optimistically before sinking again. Listening to it again I realized that there aren't enough notes in the vocal part to really make it work, but I'll hit that again when the entire structure is fleshed out.

The second 8 parts will lift up somehow, but repeat a few times. It takes a while for the singer to realize he's "saved", so I'm thinking that a repeating rising and dropping might impart that sense emotionally. Maybe an upward key change?

The last part, the joyful refrain, should soar. I think this will happen through longer notes and a very energetic playful sequence of notes, like you're on some kind of awesome theme park ride that gives you a huge boost. Or something.

So that will be the next focus; any refinement will wait to see what I have at the end of the next stage. I think it will be actually rather difficult because I'll need to keep the big picture in mind of progression, and it takes me a long time to figure out how the notes fit together. The awesome part, though, is that it doesn't take a lot of time to try things, and the feedback is immediate. When I'm doing graphics work or development, it takes a lot longer to get to the point where I can really immerse myself into what I'm making.

You should be able to hear what the progressions sound like by clicking the player button below. You can also download the general midi file or the entire archive.

Click the Play Button! It should play 24 seconds of GospelTest00 (just church chords) and then GospelTest01 (a first pass at trying to create something over the chord structure):

Mission Improbable: Making Some Gospel Music

It's been a pretty good week so far, thanks to some recent insights I've had about my so-called "work life balance". I've been continuing to take notes on Tom Hodgkinson's The Freedom Manifesto, and I've been feeling better about making choices that will allow me to relax and be happier with my pace. Up to now, I've always labored under the assumption that I needed to work faster, work harder and be optimally competent because I attributed these qualities with "professionalism" and "success". I also suspect there is some subtle cultural conditioning going on too; even though my parents didn't force me to study hard or strive to be successful for its own sake, I nevertheless picked up this value through sheer osmosis. It went without saying. My parents and the extended families all have a very highly-developed sense of mission as well, being involved with the Christian community in Taiwan and other academic pursuits. Although I didn't follow in their footsteps, the idea that there was a higher authority to which I needed to report to. God? Standards? Ideals? Philosophy? I kept seeking it out, craving some kind of closure, until sometime this past Monday. I have come to the conclusion that I will actually be pretty happy seeking things out, because I have been happy doing this. The rest will take care of itself. This is just one of those life lessons, I think, that every person has to learn for themselves. I've been told this over and over by people I love and respect, but you know how it goes: it's just not the same unless the apple falls on your head.

Having come to a kind of inner peace, my daily routine is starting to come back together after having fallen to frequent travel and a veritable parade of wheezy winter coughs. I have started going to the gym again too; I was happy to see that while my muscles are noticeably flabby, cardiovascular endurance is not as degraded. But something else was missing: with my reacquired sense of personal stability, I felt the need to do something non-routine. Something impossible, or at least relatively unexpected. My sense of creative adventure has returned.

I mentioned this to one of my coffee buddies this morning at Starbucks, looking for a suggestion of something that would be IMPOSSIBLE to do. I like impossible tasks because it's fun to think of ways to whittle 'em down to the realm of probability, winning the no-win scenario, and so forth. Unfortunately, the first thing she said was "GO TO THE MOON", and though I instantly started thinking about ways to get there, all of them took a lot of years and a lot of money; visiting the moon will have to wait until I make a bazillion dollars or Southwest starts flying there out of Las Vegas (all my "frequent flyer" miles are with them, you see). So that kind of took the wind out of it. However, on the way out I heard some gospel music playing over the cafe loudspeaker, and thought, "Hey, I should write a gospel song! How hard could that be?" Sure, I can't play an instrument worth beans, can't read sheet music, and my experience creating music has been limited to editing stock sources for use in "online webinars" for IBM...but why not?

And so, this will be my amateur project. This should be fun.

The Approach

There's a favorite line I have from the television show The Unit, which is about a team of special forces operatives. What I like about the show is that it's about the characters and their mental attitude toward getting things done, not about shooting stuff or knifing people in the dark (though they do that pretty well too). In one episode, the wife of one of the officers is trying to work out a moral dilemma, and another member of The Unit relays some advice that the commander had told him in the past:

If you knew the answer, what would it be?

I like that line a LOT, because it's tricky and if you can master it, it probably is the way to at least get going on finding the solution. In a lot of cases, there is no wrong way to start, and the solution presents itself only after you start looking for it.

In the case of writing Gospel music, I'm in a similar situation. I have an idea what Gospel music is from movies like The Blues Brothers, the occasional episode of American Idol and Ally McBeal. I once saw a volunteer Gospel group perform at the Boston Museum of Fine Arts, trying in vain to raise the roof with a pretty introverted crowd. And because I was raised first as a Presbyterian minister's son, then later grew up on the campus of a theological seminary with a very LOUD music building, I've been exposed to a LOT of hymns. On the seminary, walking to and from the school bus stop I'd hear vocalist singing scales over and over again. And when Grandpa (himself also an ordained minister) gave me a full-sized harmonica that I started to play with, I eventually discovered that it could ONLY play nursery school songs and hymns. Every time I would try to toot out something funky or popular, I would find that I was missing some notes. It turns out that my harmonica was made for the key of C Major, which is the key of practically every hymn played in mainline protestant churches. So that's what I could play. After a while, I sort of gave it up when I started to crave "bluesy notes" and didn't know that's what it was. A few times since then I'd tried to figure out how music "worked", reading up on harmonic relationships between notes and driving my Mom nuts with questions like, "but where do the notes come from?" Mom liked her notes just the way they were printed on the sheet, she did; my continual questions about the theory and the underlying principles of why some chords sounded "good" and other "ech" didn't get too far with her. This mystery continued into college, when I asked some guitar players some questions about chords, and they started happily tossing around terms like "Minor 7th" and "Diminished 9th". When I asked them what the numbers meant, they really couldn't tell me; they just knew how to look up the fingerings and believed such chords were "cool" in some musical aesthetic sense. I eventually figured out that in common use, these terms are largely descriptive of a certain pattern of notes and are used to reference characteristic features of certain genres of music (jazz, for example). The connection I was looking for was technical and emotional: I wanted to map chords to emotions, transitions, and progressions, because that's how I hear music.

So that's what I have to work with: an emphasis "feeling" and the ability to "hear" when something sounds right to me. I have a keyboard and the imprint of 18 years of classical church music shaping my idea of what music sounds like, with more recent exposure to mainstream blues, jazz, and popular music.

The elements of my Gospel song, as far as I can figure them, are something like this:

• Words There's usually something about Jesus, The Lord, Praising Him, Sweetness, Grace, Salvation, Mercy, Forgiveness, Having Journeyed, Now Seeing, Getting Shown the Way, Being Led, Seeing the Light, Feeling the Love, and the occasional acknowledgment to Joyful Noisemaking. There's a little story that goes along with it; the classic progression is being a sinner, down and out, or otherwise depressed, but then POW, Jesus or The Lord steps in and the choir has something to get excited about.

• Music This is where I'm a little shaky on what to do. Ordinarily I would want to make something original but it's quite probably that anything I come up with is going to sound like something I've heard. Compounding the problem is the fact that I can't actually play the piano. What I can do, however, is press down keys on the keyboard and work out what the progression might be, mapping my notion of how the song progresses emotionally to choose what I want. Then I can work out the interconnects between each chord later. I know, I know...I am such an engineer. I guess this is a good opportunity to learn how to transcribe notes. When I was a kid I did take some piano lessons, but never could really read sheet music as I just pretended to read, relying instead on memorized finger positions.

• Vocals I can't sing either, but I guess if I'm writing a song I'll have to try. I'll worry about that later.

• Song Structure I don't have one in mind, but what usually seems to happen is that you have a verse that goes on for a while, which leads into a refrain that sets the common theme, and then the song repeats. In popular music, there's usually some kind of change-up (I think this is called "the bridge") that is in a different key, but I'll not worry about that.

So that's the idea. I think I'll start with Music first, because I just plugged in the keyboard.

Expectations

I expect this to be fun, and to be educational, that's it. It will probably result in something that's very derivative and not very "good", but it's an experiment; maybe I'll luck out. I'm also NOT looking for proper instruction or a methodology to learn "the right way" to do this, which is part of the fun. I have a tendency to get stuck in the "optimal instruction" mindset in the first place, so deliberately approaching music in an ad-hoc fashion might help banish my stiff and unfunky mannerisms. May the Good Lord have mercy on me and show me the way! :-)

» On to Part 2

Any C# Game Developers North of Boston?

I've been slugging it out with C# and Managed DirectX for the past couple of months, and haven't made as much progress as I'd like. My conclusion: I need people to jam with to develop momentum. I am pretty much building my development process up from scratch, as it's been years since I've done this full time, and I miss having other people to talk to about technology, game design, and interactive ass-kicking.

There possibly is a group already meeting that does this; I should check out the long-running Boston Post Mortem again, our local area game developer social group to see what's going on. I figure there's got to be a few people in Southern New Hampshire / Metro Boston North--and it only really takes a few--who are motivated and skilled enough to tackle some game development topics and develop best practices. The ideal candidate already knows one of the following so they can bring some skill to the table:

• C# / Microsoft Programming Environments
• Game Development, especially tool chains and workflow
• Object Oriented Programming
• DirectX 9 / Win32 Programming

I used to work in the game industry, so I know something of the second topic though it's pretty out of date. I'm a decent-though-workmanlike programmer, something of a architectural purist that's learned to make concessions to just getting things done. I love documenting APIs for some reason. And I dig algorithms, version control tools, debugging, and Joel on Software.

If you're interested in forming a local study group, let me know and let's get together. If long distance is workable through online collaboration tools, then we can try that too. This is related to my main project for 2008 to develop a large-scale museum exhibit, so I will be working on this full time. I'm not out to create the next cutting-edge graphics demo; I just want to have a decent architecture built on some good tools.

Drop me an email via my contact form or leave a comment here. To maintain high signal-to-noise, I will classify interested parties as either a contributor or a subscriber:

• Contributors are developers with experience in one of those three areas I mentioned, and are genuinely interested in understanding the technology to apply it to their active projects. We expect results from each other, in other words, though we are all working on our own projects.

• Subscribers are people who are interested in what we're doing, but may not have the requisite time or experience to contribute knowledge and code. Nevertheless, they want to know what's possible, and maybe learn something about how they could start doing what we're doing. 3D and 2D Artists, interactive developers who use Flash/scripted environments, museum exhibit designers, advertising technologists, and experience designers would probably fall into this category. The input from this group of people is a necessary part of developing anything that has life in it, and communicating how this stuff is technically achieved will provide valuable insight. Or so I hope.

Of course, this presumes that people are actually reading this and care :-) At least I tried :-)

One Laptop Per Child, and One for ME!

One of my colleagues out in San Francisco recently told me me about the One Laptop Per Child organization's program Give 1, Get 1, in which North Americans can donate $399 to the cause. This donation will pay for one XO Laptop--what used to be known as the $100 laptop--for a child in a developing country. Additionally, you will also receive your own XO. $200 of your donation is tax-deductible. You have until November 26 to participate!

I saw the laptop at SXSW earlier this year, and at the time I thought it was cool, but didn't know all that much about it. I'm not sure if this is a production model or not (I saw it at the "Is The Book Dead?" panel). The official specs look awesome from the design and software philosophy side of things. It's small. It's rugged. It's cheap. The software is open source, and it's designed for mesh-networking in both the technological and social sense. It's a fresh piece of design, one that really appeals to me in that it puts computing in the hands of people outside of the office. It reminds me also of one of my favorite books ever, The Diamond Age, a story about a disadvantaged girl that uses a "Primer" (essentially, a very powerful interactive computer in the form of a book) to become Awesome Individuals. The OLPC is, in my world view, one of the first steps toward creating a hardware platform that can accomplish something along these lines. Oh, the machine is "underpowered" by even 5-year old benchmarks, but what it lacks in raw CPU is portability, durability, extremely low power, and outdoor usability even in bright sunlight. That's where the money went, which is eminently more useful give its intended application. This is a machine that, I hope, that can be used more like tool in the hand. By contrast, most of our creative computer-based tasking is related to correlating, compiling, and assembling bits of data into a finished product; we are not really using the computer as a tool as much as the computer is using us to steer the creative process. We shall see what happens. I would like to try using one of these for a few months to see how it changes my working relationship with data and communication.

You can read more about the OLPC and XO on the official website. Check it out!

Modern Spellbooks

As technology gets newer and I get older, learning new things becomes frustrating. For example, I want to learn how to work with MySQL for web development, program 3D games, and play the guitar, but my lack of ability in these areas prevents me from achieving my overall life goals. There's also day-to-day stuff that, as a 40-year old American male, I feel I should know: balance my household finances, invest in the markets, ride horses, and flirt with women without throwing up. These are tasks that are, from my perspective, hard to learn for several reasons: a lack of good mentors, reference materials, and classes. And that's even without mentioning the magnetic properties of my ass with respect to my couch.

When I overcome these obstacles, I still hit the proverbial brick wall; for whatever reason, my brain can't quite deal with the important task of learning before getting bored or sleepy, and I end up going to get a sandwich instead or watching Age of Love on TV.

It's easy to presume, as I join the ranks of the Newly Old, that my mind is becoming less flexible. This is the common wisdom; for example, people say that it's tough to learn languages when you're older, and that we should have done it when our minds were most facile: around the age of 6, I think. Although I don't have any studies to back me up, I'm pretty sure that other factors are greater contributors:

• We're self-conscious about not being competent in front of other adults, so we iterate less and thus learn more slowly.

• We're not particularly motivated, given that mass media tells us it's supposed to be easy. When it's not, we give up.

Since we're all grown up and have our own money, we expect to be able to buy knowledge and expertise readily. It's really amazing just what you can buy, and we've grown to expect the easy access or we get real mad. It doesn't help that our advertising, at least here in the USA, tends to emphasize the quick and easy fix. We expect instant gratification, and thus we've forgotten how hard it was to learn our first lessons, and we've also perhaps forgotten how to learn for ourselves. I wonder if kids these days even know what it's like to have to wait for anything.

A LESSON FROM THE PAST

Yesterday morning I was doing my morning coffee thing, glumly looking at all the things I wanted to do that I was unable to follow through with due to a lack of understanding. One of the main ones has been transitioning my blog to Expression Engine, which I think will allow me to more easily expand the content offerings on my website while improving overall service. I had met up with Mark J. Reeves recently for lunch recently to catch up, and asked him about the possibility of writing a web service that would save data from my Flash Apps and integrate with the Expression Engine user management system. Mark, who's a competent execution-oriented web developer, told me exactly what I needed to do: write some SQL queries to access the pertinent database tables, maybe even repurpose the underlying blog engine to store data for me in custom fields. The problem: I don't know SQL, or what tools to use, or even how to talk to the database. I am paralyzed by not knowing what the best practices are, haunted by issues of scalability and security, and most importantly of all: I was not looking forward to learning all that stuff. I could not readily apprehend the structure of the material, and therefore I could not approach it logically.

Reflecting on this experience, I found myself reminiscing about my youth, when I first started learning about computers. Today, computers don't scare me at all, and it's because I have experienced them nearly to the transistor level of operation. As a result, I can look at a computer system and "read its aura" to figure out what's really going on. That is now, but when I was in the 7th grade, computers were as mysterious to me as, um, MySQL is to me right now. I vowed that I would master the computer and learn all its secrets. Somehow.

And so I started my notebook:

Some historical notes first: this actually isn't the original notebook: it's a manually-transcribed copy. The original copy went to a kid named Derek Bumpas, which I handed to him just before I graduated; he had a good attitude and was eager to learn even though he was in the 8th or 9th grade. He recently contacted me, some 20 years later, to say that getting that notebook had meant a great deal to him, and had helped put him on his path in computer science. That was really nice to hear.

When I first started taking these notes---recall that there was no Internet, hard disks, or multi-window multi-tasking operating systems---paper was the only way to simultaneously take notes and learn. Every nugget of wisdom gleaned from hours of tinkering was transcribed, as cleanly as I could, into this notebook so I could share information with friends at school. There were all kinds of things in the book, all of them interesting to me. It was, in essence, my spellbook. Here's some of the entries:

INCANTATION

An Applesoft BASIC routine to reformat a long string so it would display nicely on a 40-character wide text display; in today's desktop publishing terms, it does "ragged-right justification" for a monospaced font. This was a common task that I had to solve in my various text-based programs, as I was pretty obsessed back then with things looking right. I finally wrote it down...my first incantation.

TRANSMUTATION

As a high school kid without any money, we often "had" to copy software. The difference was that software back then came on 5.25" floppy disks that were copy-protected using peculiar algorithms; it was a fun challenge to try to figure out exactly how to elegantly disable them by rewriting the program code. This was my real education in computer software debugging. The code listing shown here is written in 6502 assembly language, revealing the method behind the protection. By understanding the principle behind the interaction with the disk hardware's imperfections and the software code that exploited them, a copy-protected disk could be transmuted into one that was easily-copied with everyday copy utilities.

SORCERY

As I started to understand assembly language, I learned how mapping the interface between code and hardware (the "input/output", or I/O routines) allowed one to zero-in on the game logic itself. For example, say I wanted to be able to change a shooting game so I had "unlimited bullets". By looking for the specific code that read the joystick button state (e.g.: is it pressed?) I could easily find the code that was responsible for checking how much ammunition was left. And once you decode one piece of code, you can infer the purpose of surrounding code. I was able to modify the game Beyond Castle Wolfenstein (the original Apple II one from 1984) to give me a 30-round submachine gun with burst capability, and rewrote the opening story to explain why you had one in the first place. This changed the nature of the game quite dramatically. By documenting the logic behind the software and noting the location of critical routines, the granting of unnatural abilities within the game world became possible.

ENCHANTING

I attended an American high school in Taiwan, and software was difficult to find for teaching purposes. Taiwan being a rather gray area in terms of copyright, my science professors would sometimes enlist my help to help them make backups of the US-sourced software; the humidity and mold in Taiwan tended to eat disks very quickly in the sub-tropical environment. Most of the time, educational software was protected with fairly straight-forward techniques using off-the-shelf protection systems. Because they were rather generic, the same general process could be used to remove the locks with just a few steps. In this photo, I wrote down the process of disenchanting this particularly piece of software, which required the use of a specialized instrument called "Advanced DeMuffin".

THE PHILOSOPHER'S STONE

And then there were the great unsolved challenges against which I beat my head. Electronic Arts had a very advanced protection routine that was designed specifically to defeat the casual copy breaker; you needed special hardware installed in your computer to even get at the code, and then you needed to understand it. I spend many hours trying to understand just how Electronic Arts' incredibly fast boot system worked, and once I understood that I tried to trace how they were doing the copy protection. Smarter crackers in the U.S. had already done it, but it was beyond my abilities and knowledge to follow. Here's a fragment of my research on the area...to me, to be able to understand this code would be like transmuting lead into gold. I should search online to see if any back issues of Computist describe this. I still kind of want to know how it worked.

THE MODERN SPELL BOOK

It's been years since I've kept any kind of notebook like this, with the exception of my patentable ideas journal. There's so much material out there now that the task of learning is equated with finding resources: the right teacher, book, or online tutorial is perceived as the "magic bullet" that will get things done. However, what I have forgotten is that the process of distilling these ideas into a form that I can invoke at will is necessary as well. It's my missing link.

I went out and bought some of the larger Quadrille-ruled Moleskine Cahiers ($14.95 for 3), and pasted a paper label on the front of it (the picture is at the top of this post). The idea is to start recording the same kind of notes that I used to do in the 7th through 12th grade; looking back, it was a highly productive period of time for me, though I didn't recognize it then. I'm thinking of just writing down really basic things that are currently mystifying, by hand, for reference in this book. I know there are plenty of reference books and online sources that purport to do this already, but do you think any aspiring wizard would buy their spellbook off-the-shelf? NO WAY! They would be told by their cantankerous mentors to go find a sturdy book and pen, and transcribe their spells themselves by hand. Because that's the way you learn, and that's the way you bind the magic to yourself.

WHAT WAS OLD IS NEW AGAIN

Ok, you may have figured out that this whole "spellbook" thing is just an amusing way for me to start learning again. The main takeaway is this: by assembling my own book of "recipes" that actually make something, I'm much more likely to maintain some kind of focus on learning. In the past, what I've done is just read everything and picked out the main principles as they've revealed themselves. What I have forgotten is that transcribing the nuggets is just as useful. I think I probably forgot this because it's so easy to just let the actual implementation replace documentation: Photoshop files, HTML, javascript libraries, etc. I don't think this is a good foundation, because you can cut 'n paste your way very quickly into the structural equivalent to spaghetti code.

Packaging the information into nuggets as I learn, which I used to do when I was younger, may be the way for me to approach the new technologies that are making my head hurt now. As an adult I had expected things to get easier, but really they are just as hard. Fortunately, I now remember how I worked through the challenge.

We'll see how it goes. In the meantime, I'm just pleased with the way my new SQL notebook looks :-)