I'll leave the solder mask question to one of the board gurus (I rarely make boards) although I don't think there should be solder mask over pads as long as Fritzing thinks it is a connector, the pogo pins should work with or without the hole, but with a hole (and a pointed tip pogo pin as opposed to the multi contact head variety) there is somewhere for it to slide in to making alignment somewhat easier in a tight space (I assume you are using less than .1 inch spacing on the pads?). You also need to check that the pogo pins can be stacked together in the space you have (they work fine on .1 spacing and will probably do fine at least somewhat smaller) as I recall the sparkfun ones are fairly fine (I have a variety in different heads and from different sources, some of the cheap surplus ones I have are a lot larger than Sparkfun's. Just some things to thing about 
Peter