Ah man, that sucks. We use PICs and several other flavors of microcontroller plus an SBC that *ahem* the vendor doesn't always configure properly so I definitely feel your pain.
Heh, I've seen the flaky SBC show too. In fact, we initially suspected the SBC that was sending the data as the culprit, not the PIC that was receiving it.
Moving from .NET to embedded development for a 10+ year old hodgepodge platform has been a strange experience; in the old days any bugs found were 100% guaranteed to be obviously bad code, now it's maybe 40% dev derp and 60% some kind of weird errata or technology compatibility thing that only manifests itself when the code is logically fine but not exactly what the hardware wants to see.
In this case, it is a custom board that was designed in-house. If the symptoms had been consistent with something wrong with the board I would've suspected hardware right off the bat, since the board design is relatively new. But it was a device internal to the PIC that was mis-behaving; I really did not expect the PIC itself to have what I consider to be a pretty blatant bug like this.
This board has been a bitch to debug. The PIC also controls the VRMs for the SBC and other devices in the system, so bugs in the PIC code can cause random flakiness like SBC reboots (by glitching the power rails). Fun times.
The years just pass like trains. I wave, but they don't slow down.
-- Steven Wilson