Hi John!
That really depends on the setup for the Checkbook in the Checkbook EFT Payables/Receivables Options window. If the Checkbook is set to use a Single Format for the EFT File Format, the File Transfer Method set on the Vendor/Customer EFT Bank Maintenance window won't affect generation of EFT payments for the Vendor/Customer when using that Checkbook.
In the example below, all EFT Vendors will use the CCD EFT File Format for this Checkbook, regardless of the File Transfer Method chosen. It can be left as Not Specified and the EFT File will successfully generate.


If the Checkbook is set to use the EFT File Format based on the Vendor/Customer File Transfer Method, leaving the selection as Not Specified does not allow generation of EFT payments.
In the example below, Vendors will use the EFT File Format for this Checkbook based on the File Transfer Method selected.

If we leave the File Transfer Method as Not Specified, the EFT file will not be generated.
Set Vendor File Transfer Method as Not Specified

Post EFT Payment

Generate EFT File

The EFT File does not generate. There is no error either.
If the File Transfer Method is set to one that does not have a File Format selected on the Checkbook, an error will occur noting the File Format is not setup on the Checkbook and that the file cannot be generated.
Set File Transfer Method to one that does not have a File Format selected on the Checkbook

Generate EFT File

The error "The selected checkbook is not set up with a payables file format" occurs, and the file is not generated.
If the File Transfer Method is set to one that does have a File Format selected on the Checkbook (or the Checkbook is set to use a Single File Format), then the EFT File generates as expected.
Set Vendor File Transfer Method to one that does have a File Format selected on the Checkbook

Generate EFT File

Here is a support article that discusses much of this setup and some frequently asked questions:
Guidelines to follow when you generate EFT files or EFT prenote files in Electronic Funds Transfer for Payables Management or Receivables Management in Microsoft Dynamics GP
https://support.microsoft.com/en-us/help/945955/guidelines-to-follow-when-you-generate-eft-files-or-eft-prenote-files
I hope that helps explain the various setup possibilities and results for each regarding Checkbook File Format and Vendor/Customer File Transfer Method! Please let us know if you have any further questions.