I've answered this question several times on the forum.
About the fields - we take those from Mailchimp. You can however use placeholders and translate those.

For example in Mailchimp you could enter "COM_CMC_CUSTOM_EMAIL_LABEL" for the email field and then you can create a language override for that string in french and english. Then when we display the form we'll pick the translation that you've set in the language override.

As far the subscribe button is concerned create a language override for:
MOD_CMC_SUBSCRIBE

Regards,
Daniel