How to harden S3?

Is it possible to lock down S3 buckets from just serverless.yml? I have my role statements set up to grant minimal access to my Lambda functions. But the buckets require public access or else the Lambdas fail with access denied.

It looks like I can set policies on the buckets in the S3 console, but then I have to determine the IAM role that the Lambdas are running under to complete the policy and that seems unduly complicated compared to the (relative) simplicity of using SLS.