CRM 2011 introduced a very easy way to connect to Dynamics CRM instances: the Simplified Connection (MSDN: https://msdn.microsoft.com/en-us/library/gg695810.aspx).
Basically it's necessary to build a connection string instead of dealing with the specific deployment type (OnPremise, IFD or Online).

The downside of using a Simplified Connection is its weakness management of passwords containing special characters like double quotes, single quotes, ampersands.

Considering the MSDN example for a CRM Online connection:

Url=https://contoso.crm.dynamics.com; Username=jsmith@contoso.com; Password=passcode;
If the password is ;abc123 (note the semicolon) an exception will be thrown with the following message:
Format of the initialization string does not conform to specification starting at index 102. The solution for this problem is to include the password inside single quotes, the following connection string will work:

Url=https://contoso.crm.dynamics.com; Username=jsmith@contoso.com; Password=';abc123';
Assuming the connection string is builded dynamically the following code can be used:

string connectionString =
String.Format("Url={0}; Username={1}; Password='{2}';", url, username, password);
What if our complicated password contains single quotes as well? Let's consider for example the following password: ;a''bc'123
In this case the previous exception (Format of the initialization string) will be thrown again. This issue can be solved "escaping" the single quotes using a Replace:

string connectionString =
String.Format("Url={0}; Username={1}; Password='{2}';",
url, username, password.Replace("'","''"));
Please note that the escape must be done also if your connection string is stored inside your app/web.config:

value="Url=https://contoso.crm.dynamics.com; Username=jsmith@contoso.com; Password=';a''bc'123';"/>
But in this case our replace method will not work, because it will replace also the single quotes delimiting the password. In this scenario I suggest to put inside the app/web.config a placeholder instead of the delimiting single quotes that will be replaced after (for example #XYZ#):

value="Url=https://contoso.crm.dynamics.com; Username=jsmith@contoso.com; Password=#XYZ#;a''bc'123#XYZ#;"/>
Then after the connection string is loaded we do the escape and the replace:

string connectionString = ConfigurationManager.ConnectionStrings["CRM"].ConnectionString;
// escape the single quotes inside the password
connectionString = connectionString.Replace("'","''");
// replace the placeholder with single quotes
connectionString = connectionString.Replace("#XYZ#","'");
Of course this will not work if the password contains the placeholder as well, so it's better to choose a long placeholder.

When the password is stored inside the app/web.config it's necessary to deal with another problem, the case that our password contains XML special characters (mostly double quotes) because this file is an XML.

If it's necessary to encode the password the following .NET method can be used:

string xmlPassword = System.Security.SecurityElement.Escape(password);
The result for the password ;a''b"c'123 (note the double quote between b and c that will create problems if not encoded) will be ;a''bc'123, a valid string to be written inside the app/web.config.