Hello,
In Outlook, go to CRM > Options > Synchronization, look for a little link in the text "Contacts specified in your Local Data", click on it and give a look at the contacts query in there (Data Groups). Make sure it suits what you need.
Let us know if you found the solution.
Thanks,
Hugo