WHERE DOES THIS MATTER COME FROM?

Massive Black Holes
The matter in the host galaxy gradually sinks in toward the nucleus of the galaxy, fuelling the black hole. The luminosity is strongly dependent on how much fuel is available.

Stellar Black Holes
Accretion from interstellar gas? NO!
A clue is given by the Algol Paradox