It is narrated from Anas رضي الله عنه that the Prophet ﷺ  came to Madīna at a time where the people of Madīna had two days in which they would play around, relax and celebrate. He ﷺ enquired, “What are these two days?” They said, “We used to have fun during these two days in the times of ignorance.” The Messenger ﷺ said, “Allāh has exchanged these two days for you for two others, the day of Aḍḥā and the day of Fiṭr.” This is narrated by Abū Dāwūd. [Mishkāt Al-Masābīḥ #1439]

Shaikh Abdul Ḥaqq Muḥaddith Dehlavī رحمه الله said in his commentary on Mishkāt Al-Masābīḥ:  “He ﷺ intended by these statements, that the true time of celebration, happiness, and bliss for the believer should only be in worship. The ḥadīth contains a prohibition about wastefully spending one’s time and fooling around, along with a subtle indication that it is okay to have some fun on the two days of Īd, as long as it is not indecent or outside the norms of the Dīn or its signs. So, take heed! It also prohibits one from honoring and being happy at the celebrations of the polytheists and their festivals. Some scholars have gone to the extent of passing the verdict of disbelief upon those who commemorate these occasions of the disbelievers. They did this to harshly reprimand such actions and to close the door of other things it could lead to as well as protecting oneself from associations of disbelief. (This) has been mentioned in Fatḥul Bārī.”

[Lamʿātut Tanqīḥ fī Sharḥi Mishkāt Al-Masābīḥ Vol. 3 Pg.556]