We study the addition energy spectra of phosphorene quantum dots, focusing on the role of dot size, edge passivation, number of layers, and dielectric constant of the substrate where the dots are deposited. We show that for sufficiently low dielectric constants (ɛ_sub<4), Coulomb blockade can be observed in dot sizes larger than 10 nm, for both passivated and unpassivated edges. For higher dielectric constants (up to ɛ_sub=30), Coulomb blockade demands smaller dot sizes, but this depends on whether the edges are passivated or not. This dramatic role played by the substrate is expected to impact the development of applications based on phosphorene quantum dots.