Like others have said…choose quiet horses known to be less reactive, be aware of your situation/surroundings to be more prepared of a potential spook (not saying you should be on edge, but if there is construction going on next door, ride more “actively” than not), and work on exercises that help you become strong and secure in the tack. Make it a habit to do no-stirrup and no-hands work to help you become more solid and stable in the tack (with a good instructor on a lunge line)- practice no-stirrups/no-hands going from walk to canter, canter to walk, etc. (so you get used to having less to rely on while you change speeds- maybe that’d help in case of a sudden spook?).
I think the worst instructors for beginners or timid riders are ones that don’t emphasize having a strong position all the time. For example, I have a friend that has only been riding on and off the past couple years. Her first year of riding, she was given a false confidence by being put on ancient, quiet, well-behaved half retired school horses and allowed to go canter on trail rides in shorts, sneakers, and no helmet in a western saddle (while being encouraged to just hold the horn). Sure, lots of fun for the new rider because there are no limitations! She can hop on and go!..fun until she hops on one that isn’t so saintly quiet and has no basic skills or any strength, and promptly gets dumped, then injured, then scared.
I’m not saying you are this rider (and maybe a lot of the advice above doesn’t apply, but I don’t know your background), but make sure you have been taught (or are being taught) good useful skills. Know what a good position is to keep you as secure as possible in the saddle (and to KEEP you in the saddle!), and learn and practice exercises to strengthen this position. Know what to do when you sense a horse acting up or becoming tense- sometimes you can see it coming and have an opportunity to refocus the horse to prevent the spook, but not if you have no idea HOW to feel it and refocus the horse.
But of course no matter how much you try to minimize the risk, horses are large, powerful animals with a strong flight instinct and sometimes we just come off. Only you can weigh the pros vs. cons and decide for yourself whether it’s all worth the risk.
Hope some of that is helpful- good luck!
