If you're "secular" why be Buddhist at all? Why not just be a secular humanist? Honestly, there is a lot of very common ground between Buddhism and secular humanism. What Buddhism offers is a way to inner peace that you don't find in secular humanism.
If you've ever been exposed to cognitive behavioral therapy, you know that this is a modern therapy approach that focuses on changing internal beliefs so that the patient no longer upsets or distresses themselves. In many ways, the Buddha was likely the first cognitive behavioral therapist. He emphasized that folks should look inward. They should examine their beliefs and realize how temporal and fleeting they are.
For me, Buddhism is all about stopping the cycle of upsetting yourself. It is about finding internal peace and happiness. That's why I often call myself a "secular Buddhist."