SecretSquirrel wrote:just brew it! wrote: While dealing with these kinds of obscure bugs comes with the territory in "bare metal" embedded work, I really like the sense of control you have over the hardware. Everything is laid bare, oftentimes all the way down to the level of individual signals on physical wires. You're dealing with fewer "black boxes"... it's just you, the hardware, and the (admittedly sometimes buggy) development tools. I'm also starting to think that my move "down-stack" is a good thing at this point in my career; a number of factors seem to be contributing to a shortage of engineers with solid experience in this sort of low-level software development.
I agree whole heartedly. I greatly prefer low level embeded work. As far as the lack of skilled engineers go, I think that is actually driven by the prevelance of high power hardware. Low level embedded development, these days, seems to be defined as using a Raspberry PI (BeagleBone, etc) running Python. There do not seem to be a whole lot of degree programs that teach low level development on regular PC hardware, much less on bare microcontrollers. The engineers that had no choice buy to develop in PIC or 8051 assembler are aging out of the workforce and are not, seemingly, being replaced.
Indeed. It is tempting for a product designer to "take the easy way out" and just throw hardware and a full Linux/Android (or even Windows!) stack at the problem! But this can have its own costs in complexity, security, reliability, maintainability, and a general reliance on too many third-party "black box" subsystems. And unless you're using an off-the-shelf RPi, PC, or similar device, someone still has to deal with the BSP and initial board bring-up.
FPGA and soft-core CPU also seems to be on the rise, now that FPGAs are fast enough to permit soft implementations of reasonably capable 32-bit CPU cores. A lot of the same design principles formerly used for hard-wired embedded systems carry over to the new FPGA soft-core world. Having also done a little VHDL development a few years ago (even further "down-stack" than what I'm doing now!), I think I may have a leg up on this niche since I am comfortable with both sides of FPGA development.
SecretSquirrel wrote:I also thing this is contributing to the general flakiness of our automated devices. When you use an bare microcontroller to run your coffe machine, you can pretty well -- with a bit of though -- account for every possible input/error condition and handle them in an appropriate way. When your coffee maker has an init file, internal message queues, and boots an RTOS,
things are going to fail in very strange ways.
I firmly believe that once the complexity of a software system gets to the point where it is impossible for a small team (preferably one person!) to understand *everything* that is going on from the user interface and other external I/O all the way down to the hardware, there will inevitably be some things that fall through the cracks, especially when development is constrained by time and/or budget. (And if you think there are projects that don't have time and/or budget constraints you are either smoking something, or an ivory tower academic.)
This reminds me of an incident about 10 years ago, where our (somewhat new at the time) Chrysler mini-van decided that it wanted to blink the headlights every couple of seconds. It didn't matter whether the ignition was on or off, or what position the headlight switch was in. It was just going to keep making the headlights blink, damnit! Everything else in the vehicle appeared to still be operating normally. After much head-scratching and Googling, I finally just disconnected the battery cable for a few minutes, effectively forcing a cold boot of all onboard systems. I never did figure out what was wrong, or what the on-board computer systems might have been trying to tell us (I suspect it was just the equivalent of a BSOD from the microcontroller tasked with operating the headlights...), and to this day the problem has never occurred again.
SecretSquirrel wrote:That said, I am considering moving over to a Rasberry Pi for my project. I'm weighing the trade offs of having to maintain a full Linux environment vs the ease of development, especially for the UI, and the ability to have real a properly multithreaded environment.
Yup, there's something to be said for leveraging all of that existing code. The correct answer is really a case-by-case thing!