No, it is currently not possible.
The GitHub integration requires more permissions than only being able to git push. It needs to fetch the Pull Requests (PR) and interact with them, among other things, and those require an API token. If these permission requirement change, we'll review adapting the integration.