Bring-Up

"Bring-up" is the term that is used to describe the process of getting a new board to function. Believe me, they don't all work the first time. Bring-up is how you coax the board into life, figure out what bugs it might have, how you can work around those bugs. Obviously, you want to get as much as you possibly can out of a test board. The worst thing is a problem that stops you dead, requiring a hardware revision or "re-spin" to make more progress. Problems that have workarounds are fine, even if the workarounds are ugly. The workarounds just need to allow you to make forward progress without spending time and money on a re-spin.

Bench Testing, Phase 1

Testing always starts on the bench using as little of the final system as possible. In this case, I could do a lot of initial testing with my bare board sitting on the bench, not even plugged into the ECU.

The first thing to do is use a meter to Ohm out the power pins to make sure that there is nothing mega-stupid like a short between power and ground. Believe me, it happens. This time, things looked good.

The next thing you do is to connect the board to a good bench power supply. By 'good', I mean a clean, accurate supply that has an adjustable current limit. The point of the current limit is that if the power supply detects that whatever it is connected to is trying to draw too much current, the supply automatically lowers the power supply voltage to a point where the circuit draws no more than the set current limit. The point is to set the current limit to a level that makes sure nothing bad will happen to the board even if it is trying to draw way too much current.  In this case, I was expecting the board to draw around 25 milliamps so I set the current limit to about 40 mA and powered it up. The first good sign was that there was no smoke, so that was a plus.  The power supply said that the board's current draw was within spec, so that was good too.

Naturally, I decided to go for a "home run" and connect to the processor with a hardware debugger. In my mind, the ability to test operation using a hardware debugger is critical. It gives me insight into what my sofware is doing right down to the instruction level.  Sadly, the debugger was a total fail. It just could not communicate with the processor's debug silicon unit. Naturally, it made me worry that the processor was in bad shape.  To get an initial sign of life from the processor, I used the BOOTSEL button method to manually put the processor into bootloader mode. In that mode, if you connect the processor to a PC via a USB cable, the PC will recognize the processor as a USB mass-storage device, looking just like a small USB flash drive. To my delight, that worked. That meant that my processor was indeed alive, along with my USB connector wiring, my crystal circuit, and even the BOOTSEL button itself. It was just the debug port that was messed up.

Given that the bootloader was working, I loaded a simple LED blinker firmware into the device.  After flashing, the board would disconnect as it should, and the LED would blink. That was a good sign.  But if I power-cycled the board, it would not start up again: there was no more blinking.  Sometimes, it ran, typically not. I checked all board voltages and crystal waveforms and everything looked fine. I googled the situation and discovered other people having the same issue with their own builds.  It turns out that some crystals have slower startup times than others. My code was using the default situation where the processor assumes that it has a fast-starting crystal. I changed that option in my build to having a slow-starting crystal, and everything worked perfectly after that: my code would load and run just fine via the USB bootloader interface, and power cycles were no problem.

Working though the rest of the circuitry on my board, I found two more hardware issues. One was more of a design issue. I had originally thought that the wifi processor should have a say in the  control of the address buffer enable. But for bringup, there was no wifi processor in the system.  I hadn't even started to write the software for that processor, so it was going to be an annoying complication to just test the EPROM processor. And that made me think that it was a bad idea for the wifi controller to have so much control in managing the address enables in the first place.  If the wifi processor ever broke or had a serious bug, you could be stuck at the roadside with an ECU that would not boot. So that put me on the philosophical path that the whole design should continue to act like an EPROM even if the entire wifi processor was dead or missing.  Data logging might not work, but at least you would get home again!  Fixing that issue was an easy change: I removed a pullup resistor and added a pulldown in its place to permanently enable that particular bus buffer enable (it has two enables).

There was one other problem: the ECU processor would not boot, even after my software released the RESET signal. That issue was worked around by removing another pullup resistor. Easy!

That was the sum total of all problems.  There were no hardware workarounds requiring circuit board trace cuts or jumper wires. It may have been the smoothest board bring-up I have ever done!

The biggest issue was that the software debugger connection to the RP2040 remained utterly and defiantly dead. That was going to be a total showstopper for code development and testing! It took me about a day to figure out that I had wired the debugger’s clock and data pins backwards to the attachment points on my circuit board. The silkscreen pin labeling was correct, but the wiring was not. In any case, it was trivial to swap the wires used by the debugger to connect to the board, and presto: I was talking to my program running inside the RP2040. Heaven!

Phase 1 of bench testing was complete.  It was time to mount the board in a real ECU.

Bench Testing, Phase 2

Phase 2 would install my new board in an ECU to see what would happen. This is still bench testing, but things are getting serious now. The hardware debugger is in the little white box at the top of the picture, connecting to my fake EPROM chip with the violet/black/green wires.  The big ECU connector is wired to my bench power supply. The little connector to the left is the one that connects the motorbike's sensors to the computer. There are no sensors attached, so the ECU software will be pretty unhappy about that, but even so, the very fact that the ECU is unhappy with its sensors will prove that my board is doing its job: serving EPROM memory cycles to the ECU's own processor. 

Because my fake EPROM is actually a full-on computer, it can do more than just pretend to be a memory. I started off by modifying my EPROM software so that it would log the first 100 memory bus accesses that the ECU processor made after it came out of RESET. That would allow me to watch the ECU processor's initial boot up sequence.

The first huge moment of truth was plugging the board onto my bench ECU and powering it up.  Using the debugger to dig through the ECU bus access log, I could see that the ECU fetched its reset vector and then started grinding through its boot sequence, one address at a time:

The bootlog showed that both EPROM reads and log writes were doing exactly what I wanted. The ECU processor was booting just fine! Everything was looking so good, there was only one more thing to do: put the modified ECU onto my bike. I went down to the garage and plugged it in.

I turned the ignition key on and heard the fuel pump prime and then shut off about 3 seconds later, just like normal. Since the fuel pump is controlled by the ECU firmware, it meant that the ECU had to have properly executed a few million instructions to get to the point of shutting the pump off.  That was another good sign.  Time for one last critical test: starting the bike. Honestly, I would only have been surprised if it didn’t run.

And it didn’t. 

Oops, the kill switch was on. With the kill switch off, I thumbed the starter, and it fired right up. That made me very happy! I celebrated with a short video.

TuonoRuns.mp4

New Features

Because it is impossible for me to leave well-enough alone, I modified my EPROM firmware to add two more features.

First off, the idea of logging the ECU bus activity during bring-up was so useful that I decided to make it permanent. Now, my fake EPROM continuously tracks the most recent 16,384 bus accesses by the ECU processor. If the ECU processor ever has trouble, I can look through that history log and get some insight into why things went bad.

The second feature was based on the fact that the ECU processor only has 512 bytes of RAM. That's not much! My fake EPROM processor has 264K of SRAM, or over 500 times as much! The variables in that tiny 512 byte memory are shared with the ECU processor's stack.  The original Aprilia firmware shows evidence of stack paranoia. When a stack gets too big, it can corrupt the memory beside it. Stack corruption problems are extremely hard to debug. To solve that, I had the idea to create an 'MROM', a "Mostly Read-Only Memory". I took a small section of some unused EPROM address space and turned it into RAM. Then, I moved the ECU's stack into that new area of RAM. Now all 512 bytes of ECU RAM can be used for other purposes, and the ECU stack lives out of the way where it can't cause stack corruption issues if things ever get out of hand. Because the whole EPROM memory is just a software construct inside another processor, it was trivial for me to do. I am betting that the original engineers who wrote the ECU code would have really enjoyed the extra memory!

Since then, I have worked on the build system for the fake EPROM.  That was a multi-month adventure, trying to get CMake to do what I wanted. Persistence can be a good thing though. Using my Visual Studio Code IDE, I can push a single button to rebuild every part of the system:

The BSON image library ends up getting stored in a specific partition of my RP2040's serial flash.  The cool thing about using BSON to hold the EPROM images is that I can combine an EPROM's binary image with things that describe the image. This other information can be things like what year and model the original EPROM targeted, if the EPROM supported multiple spark and fueling maps, what kind of mods the EPROM is designed for such as stock pipes, slip on pipes, or full race exhausts, or even if a specific EPROM needs to run using a software version of a copy-protection daughterboard or not. The BSON objects would even allow me to tag the individual images in my EPROM image library with my personal observations. For example, to remind myself with what I liked or didn't like about a specific EPROM. Those observations become part of the library so that I can't lose them.

One other general feature of the build system is that it can determine if a specific stock EPROM is compatible with my special logging firmware (most are, but not all). If the EPROM is compatible, the fake EPROM processor code can combine all of the spark and fueling maps from the compatible EPROM with my own logging code. Basically, it means that any compatible EPROM instantly gains data-logging capabilities.

One of my goals for the coming year is to convert the project's github repository from being a private to being public. I'm not trying to make a product or make money or in fact, do anything except have fun working on what I think is interesting. But there might be other Aprilia owners out there that also want to have some fun.  We'll see!

Next up: A never-ending "To Do" list