Social engineering is a numbers game, so the fewer scam calls, the fewer potential victims. However, with data breaches constantly occurring, your phone number may sooner or later get into the wrong hands.
https://cybernews.com/security/voice-scammers-getting-smarter-set-up-phone-defenses/