Until your child is big enough to disarm you, you really don't have a much to worry about except when you are asleep.
Keep your firearm on your body or locked up in a good solid safe with quick access for those times you hear a bump in the night.
Well, there is one thing to worry about...that time you are in a crowd of people and your child announces to everyone, "My daddy is carrying a gun!"