How do you prevent users from accidentally seeing the sharepoint site?

I've seen this power platform question in the forums.

Solved: Hints and tips using SharePoint as a Datasource

Its marked as solved but it doesn't really much answer the OP's question. So technically if you grant users access to the list they will be able to see it in their "recent site" whenever they access SharePoint. Has anybody found a workaround for them not being able to access the site at all but still has "contribute" and "read" permission to the lists? I initially thought that they wont be able to enter the SharePoint site because they are not invited as a member even though they have some permissions in the list.