hello, here to share:
I power my arduino nano directly with 5V or lithium battery power, as well as the SD card.
most they works fine, sometimes (<2%) the card went hot and needs restart.
if you have trouble with level shifter , skip it and try?
note that some SD adapter has pull down resistor over level shifter,
if you don’t insert a SD and the pins were floating, pull-downs were still likely to interfere your SPI port, in case you use other SPI device.