See here (page 11) and in Notes on Nonequilibrium StatPhys MT2015 Oxford (mostly stochastic processes) (page 42). Generalization of Lagrangian multipliers in finite optimization problems.