Hi Leung,
Since the security model of Dynamics 365 is based on 4 levels (Users, Business Unit, Parent: Child Business Unit and Organization) of privileges for entities and not records, the only way to define a specific record as private, would be using the structure of Business Unit on Dynamics 365.
Example: Basically, your boss should be at the root BU and all other users in a Child Business Unit and after that, all roles should not have Read privilege for Account/Contact entities considering Organization level. Remember that System Administrator with Read-Write Access Mode has this privileges by default.
Considering OOB features, I don't think that other security features like roles for forms, field level security, position and hierarchy and teams could help you in this scenario. If this Contact records are really personal, maybe your boss should reevaluate to keep it inside Dynamics 365.
For more details about security concept inside Dynamics 365, please check this article: https://docs.microsoft.com/en-us/dynamics365/customerengagement/on-premises/admin/security-concepts
Thanks,