Outstanding success! I figured out the problems I was having getting PORTC’s pins to Respect My Authoritaaa…
Turns out that JTAG is enabled by default on this device, and for some reason disabling JTAG internally didn’t work. I tried writing the bit by assigning it twice in succession in gcc. Instead, I changed the fuses to disable JTAG, and that cleared up the problem.
An interesting side note about how I realized it was the JTAG settings getting in the way: I was thinking that perhaps some alternate function of the PORTC pins might be getting in way, but it didn’t dawn on me which function until I ran a simple sequential count on the pins in fairly rapid succession, using the STK500 LEDs to show the output. Lo and behold, only the least- and most-significant two bits changed state. Checking the docs, sure enough, the remaining bits were all dedicated to JTAG.

Once I got PORTC in order, I reconnected the LCD which I had wired up a couple nights ago. It worked like a champ, with the otherwise untested code I wrote then. This image shows the LCD displaying my friend’s name and the current encoder count (more on that below).

LCD Display

After the success with the LCD, I couldn’t resist getting at least one wheel encoder working. So I took a stab at setting up an AVR interrupt handler, set a few configuration bits, enabled interrupts, and presto! It worked.
I’m taking this approach to the quadrature decode: I feed the “A” channel from each encoder into one of two interrupt-generating inputs on the ’32, which is configured to interrupt only on rising edge transitions. Then, in the interrupt handler, I check the state of the “B” channel for the corresponding wheel, and if it’s low, I subtract one from a 32-bit counter, otherwise I increment the counter.
A little impromptu testing (marking the tire, counting a few revolutions forward and the same number back, should get a result count close to zero) seemed to indicate very little error (no missed counts) using this scheme. There’s enough resolution in this encoder that it registers a step even in the play present in the gearbox.
Another check was to see how long the ’32 spent in the interrupt handler. The image below is a shot of the oscilloscope. The top trace indicates the low-to-high transition of encoder channel A, and the lower trace roughly indicates the time spent in the interrupt handler (I set a pin high first thing in the interrupt handler, and clear it as the last step before returning). The handler is much faster than the encoder half-period, and this is only on a 4 MHz prototype. (The larger image has some annotations).

Interrupt Handler Timing ’Scope Image

Okay. It’s very late, and I’m going to pay for my stubbornness in the morning. Here’s an image of the bench after tonight’s fun. The colorful board is the STK500.

Workbench after Encoder First Success