Parallel processing
17/03/2026
Each MID outputs 1 pulse per Wh. The MID monitoring the overall power consumption is the one that will see the heaviest load. Yet the installation is capped by contract at 6 kVA. Thus the maximum pulse rate is only 1.67 pulses per second. Rather low, you may think.
However, any MID can output a pulse at any time. Even if the odds that the 38 pulse sources (35 MID meters + 1 overall electrical consumption meter + 2 pulse sources for water and gas) fire exactly at the same time are certainly very close to 0, it still can occur that more than one MID outputs a pulse at the same time. So we have to deal with up to 38 simultaneous pulse inputs.
That's where the use of Tiny2040 comes in handy. Each tiny can handle 12 GPIO (General Purpose Input/Output) pins that can be set as input (with a very convenient pull-up resistor) and each has its own interrupt line. Only one ISR (Interrupt Service Routine) can be defined and that's also convenient to factor the code. That ISR receives the GPIO identification as one of its parameters; it can handle each interrupt separately. Written in C and kept to the bare minimum of incrementing a RAM pulse counter for each tiny, the ISR will be run almost instantaneously, in a fraction of a pulse duration. The main tiny core is dedicated to running that ISR, waiting between calls by flashing a blue light every second.
That's the first layer of parallel processing: recording pulse-counter changes for each MID.
The second tiny's core looks for pulse-counter changes every 420 ms. If any change is spotted (there can be more than one), it writes the list of (pin, counter value) to the USB link with the Raspberry Pi.
That's the second layer of parallel processing: reporting pulse-counter changes for each tiny.
The Pi runs a distinct thread for each tiny where it looks for incoming data from each tiny. It's too complicated and not worth tracking a real-time clock on each tiny, not to mention synchronising all clocks through each tiny and the Pi. So, it's this thread's task to link each pulse to a date and time value. Doing it within the Pi moreover allows us to inherently have a common time reference for all events. That reading thread process does the minimum: it records the values it receives in a SQL database. Also written in C, it will run very shortly after receiving the data from the tiny, effectively linking the correct timestamp (with a precision of one second) to each pulse. These processes are run by a service at the operating system level. Its main thread lists all connected tinies, launches one thread for each and waits hopelessly for these threads to end.
That's the third layer of parallel processing: recording pulse-counter changes in a database.
Computing the instantaneous consumption from pulses emitted once a given consumption threshold is reached requires two timestamped pulse reports (a counter change may represent more than one pulse). That implies a conversion of counter values to pulses. This is done every second when the web interface is viewed and every hour by a cron job to avoid accumulating too many counter-value changes. The same code, of course written in C for sheer speed, will handle that conversion in a fraction of a second.
That's the fourth layer of parallel processing: computing timestamped pulses.
Of course, the Pi runs several other processes, at least for the web server that provides the web application for the HMI (Human-Machine Interface) where the data is presented and analysed.
Altogether, there are more than 14 processes running in parallel for this test project's context. Those processes are carefully written to smoothly handle edge cases and scale gracefully; they will withstand the accumulated pulse counts over time. The Pi used in this project is a model 5 with 16 GB of RAM and a NVMe disk. It will handle the load with ease for this test project and could handle much more complex contexts.
Needless to say that all this parallel processing requires careful handling of concurrent events.