Specific user to a specific folder

How do I do these?? I want to restrict upload access to some user to a specific users. For example I have 5 intermediate users and 5 folders for each users. other users must not be able to upload to the other folder but they can access them and download them.