You do not need to use FBA or be a part of the Seller Fulfilled Prime program to offer free shipping on your products.
Free shipping can be offered by modifying shipping settings in the Amazon Seller Central account, section Shipping settings.
For more details, please refer to this guide on how to create, edit, or delete shipping templates.