Starting location bug

When I try to navigate and use my current location as the starting location, it instead seems to always choose my home or work location as the starting location. If I logout it works fine.

Elderly Gentleman about Google Maps, v4.31.1