Hi,
Sorry but there is no WYSIWYG editor to create popups. You can use HTML/ CSS page and add them as a web resource in CRM. Once added, you can add you own custom button and open you web resource. With this said, it is not straight forward and personally I don't think it is worth an effort.
The simple solution would be to remove button completely and have a flag field instead on the form say "Resolve Case?" probabaly on the closure stage which display the field you need and then once save, you create a workflow to resolve the case.
Hope this helps.