The primary risk is that someone else files first. In a first-to-file system, priority goes to whoever files the application earliest. Every day without a filing is a window in which a competitor or bad-faith filer could establish priority over your mark in that jurisdiction.
In the US specifically, unregistered marks do carry some common law rights based on actual use, but these are limited to the geographic area where the mark is used and are significantly harder to enforce than registered rights. An unregistered mark does not give you the right to use the ® symbol, to oppose a similar trademark application, or to access Amazon Brand Registry.
Filing an intent-to-use application in the US, or a standard application in most other jurisdictions, secures priority from the filing date regardless of when commercial activity begins.