OAuth 2.0 - Cross Site access to REST

From my experience the only way to retrieve an OAuth token from REST, from an external client is to use authentication details of a user that is in the RemoteService role when posting.

POST: https://MyImisDomain/Asi.Scheduler/token

Otherwise, if I remove the user from this role, even for MANAGER, I see the error

{
"error": "invalid_grant",
"error_description": "Not authorized for Services."
}

Is this correct, do I always have to use the RemoteAccess role or I have I missed something?

What rights does the RemoteAccess role grant?
From my brief testing it appears to grant access to other unrelated party's data.

Is there a way to use REST, cross site, where regular public user authentication can be used and the ClaimsMapping / Templates are honoured?