I see, this was handled for binomial deviance by the 'ylogy' method, which computes y log (y / mu), defining this to be 0 when y = 0. It's not necessary to add a delta or anything; 0 is the limit as y goes to 0 so it's fine.
The same change is appropriate for Poisson deviance. Gamma deviance looks like it also has this issue but I suppose it isn't defined at 0 anyway. I don't know if implementations still try to return something that isn't NaN or what here.
Anyway, I think it's fine to open a JIRA and PR to make that change.