Is there any logic to control a field like when i am going to create a new case from case subgrid + button then an kind of alert message will pop-up and if we click on next then i am able to add/ make a new case otherwise not?
You need to customize the ribbon button for that, call java script function to alert, make sure you have the functions in the order, so your script gets triggered first and then the default functionality of opening case form.