Restrict access to File Manager

How is this not planned and implemented yet? This is super important and seriously dangerous that anyone can download all files.

This is a major security issue.

Folder and File permissions need to be granular. The ability to download ANYTHING should be restricted by default with the option to assign read only / download permissions to the user/team level.