we experienced production of metrics obviously failing (no metrics produced) without any error log reported. The production would work fine in local mode, only remote mode was failing.

This seems to be an issue with the worker mechanism. Temporarily fixed by aligning the version of some dependency packages et restarting the production workers (don't know which one of action did the trick): must be improved by better error catching and reporting.