Yes you can use the Access teams to implement this requirement.
Register your plugin on this field change and after that add/remove users for this specific record.
i.e. Access teams helps to maintain the security on record level.
Thank you,
Amit katariya