Hi
I would recommend you enable Server-Based integration as that's the recommended approach. Two key benefits are, the users don't have to login to SharePoint to see documents in CRM and the out of the box documents tab renders much faster compared to List based approach.
Please follow the link below to see if you Sharepoint meets the requirement to enable Server based integration and also to see what you need to do to configure Server based integration
[View:https://docs.microsoft.com/en-us/dynamics365/customer-engagement/on-premises/on-prem-server-based-sharepoint-on-prem:750:50]
Few other useful links
[View:https://www.cloudfronts.com/enable-sharepoint-for-dynamics-365/:750:50]
[View:https://docs.microsoft.com/en-us/dynamics365/customer-engagement/admin/set-up-sharepoint-integration:750:50]
Can we have one document library for each entity?
Yes you can document library per entity (Non-entity based on configuration) or you can have records stored in folders under Contact / Account (Entity based configuration.)

Or do we need to specify document location for each record in the entity?
No CRM will automatically create folder for each record in Sharepoint and also create the document location in CRM when the user visits the Documents tab the first time.