When a battery discharges the current exits through the positive terminal at a higher potential than it entered.
When a battery discharges the current exits the battery through the positive terminal as a circuit connected to that should conduct current from positive to a negative potential.
Accordingly, the current enters the battery at its negative terminal.
In practicality, batteries contain some internal resistance that restricts the current flow during current flow.
That is why there is a potential difference between the 2 terminals.
To drive the current through the external circuit connected to the battery, it keeps a higher potential at its positive terminal while the negative terminal is kept at a lower potential in contrast.