I don't see anything wrong structurally with the argument, but I think the whole context of the argument is unnecessarily narrow.

For instance, what about people who opt to donate to organizations that provide services for the homeless rather than giving money to individual homeless people? What about people who work on activist projects to make politicians take more proactive and beneficial actions for the homeless?

I guess I'm saying that there are ways to help homeless people besides giving them money.

Insofar as being "morally worse off", I interpret it as the psychological conflict of doing something that one knows is "wrong", and the stress that can cause in the person's psyche, which in turn can harm a person's physical health, not to mention impact the health of their relationships with others.