Permissions against the Bucket are separate from permissions against Objects within the Bucket. So you have to grant permissions to both.
Write this code to combine bucket-level and object-level permissions:
{
  "Id": "BucketPolicy",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllAccess",
      "Action": "s3:*",
      "Effect": "Allow",
      "Resource": [
         "arn:aws:s3:::my-bucket",
         "arn:aws:s3:::my-bucket/*"
      ],
      "Principal": "*"
    }
  ]
}
I hope this helps.