I am wondering, it seems talking to sphere over the serial from something like an RPi, would works but is it possible to speed things up for a more real-time response or would some kind of custom firmware be needed?
I’m not sure exactly what you’re asking. First off, you posted the question in a thread about RVR firmware, so I’m assuming you’re working with RVR. The RVR firmware acts on most commands within a few milliseconds of reception on the UART, depending on the schedule of the task that handles it. This is much faster than the EDU app sends commands over BLE.*
Are you just concerned about the execution overhead of Python running on Linux? The last time I measured execution time of the Raspberry Pi python SDK sending back to back drive commands I was also doing some console printing and saw 10 ms delays between outgoing commands. I suspect most of that was from the console printing, and again, this is much faster than a BLE connection with the EDU app.
If you don’t like the non-real-time nature of Linux and want to go bare-metal or use an RTOS on an MCU dev board, take a look around the forums at what other people are working on. I know @rmerriam was putting together a C++ library for RVR communications that can be adapted for a wide variety of embedded platforms. I think there are a few other similar projects out there.
*I should mention that BLE averages out to similar bandwidth to the 115200 baud UART connection, but with more variable latency and the possibility of dropped packets. EDU is not able to use the maximum bandwidth of RVR’s BLE connection because the actual throughput varies wildly between different mobile devices.