When you say something in an inappropriately casual way, you do it flippantly. Try to avoid talking flippantly about ghosts and monsters around your young cousin, who is terrified of spooky creatures.
It's usually fine to be casual, but when you talk or act flippantly, you're likely to offend someone. The word implies a lack of seriousness or respect, so if you speak flippantly to your teacher — answering a question with a snarky "Whatever!" for example — you might just get yourself in trouble. Flippantly comes from the adjective flippant, originally a positive attribute; in the 16th century it meant "limber or nimble."