That is correct, you need to remove the px from the svgs (Inkscape apparantly adds them due to css). It isn't only svg export that breaks if you edit a part with px in the font-size usually by the second time it reduces the font size to 0. This behavior is documented in the parts-file-format (along with the recommendation to remove them) here:
Peter