Yeah, that GitHub is a bit unclear.
If you click on the actual file in the directory it gives you the download on the right, or click on "View Raw" when the file is open.
If there is no download I usually click on the raw or image, and right-click save. For some reason one of my browsers corrupts downloads this way, so I have to use another.