We don’t specify rifle or pistol for lowers in TennesseeIt can be proved that you assembled a pistol from a rifle even if you don't "tell anybody" - so I would not say it will never matter. People have been caught and prosecuted for doing this.
If the lower were sold assembled as a rifle, the 4473 will state "rifle" as well as the manufacturer's/distributor's/dealer's records. If you assemble a pistol from a lower that was sold as a rifle in the paperwork*, you can be caught, charged and convicted of a federal gun crime - it has happened. The SN, documentation as originally a rifle and the pistol configuration as discovered is all that is needed to prosecute.
What are your chances of being "caught" ? I'd say it depends upon where you live and what you do.
*Even if it was a lower only, with "rifle"marked on the 4473 - which would be incorrect - but some FFLs have done this. When sold as a lower only, it should be marked "other".)