I support this request and I hope it will not need much development effort!
If I may add, I would suggest to default to following process: when user is logged in, use the registration address as the default delivery address (still check against de (geo-)location requirements) and offer the drop-down list still where the user can select alternative delivery address before order-confirmation.
In my current test-phase with regular customers/end-users, most questions come from the awkward logic during the check-out process. Basically I would appreciate that a successfully processed first order becomes the default for each returning known login. It would be useful (but not absolutely required) to have the option to mark such known users in the user-admin section as confirmed working delivery address.
It’s like this also in real life: returning customers come to the restaurant, they know the front-door to enter the restaurant and they know where to get the menu to select their order.. It is always the same! (well, in most cases) 😀