I asked someone about text and this is what I got, which would explain the default px.
FWIW, this may go back to the way units are handled in Inkscape, svg and css. AIUI, svg does not require explicit unit identifiers (e.g. px) but css does.. so to be css compliant, Inkscape adds the identifier. ( I could be way off on this, maybe a dev can chime in.)
There was a point I think, when setting the text units in preferences also wrote the selected unit identifier in the xml (e.g. pt), but that was in a development build.
Ever since I fixed my icons in Ink's XML Editor the File/Document Properties/Default Units change is more global. Basically everything changes except stroke.
The only way I could change default units was use a template.