CT Weirdness

Only a quick thought about another word related curiosity that I popped in my head earlier.

What is the deal with the suffixes -er and -or for verbs ending in -ct to describe an object performing the specified action?

Here are some -ct verbs that use the -or suffix.

act -- actor
react -- reactor
extract -- extractor
elect -- elector
collect -- collector
eject -- ejector
inspect -- inspector
abduct -- abductor

But, what puzzles me is how come some other -ct verbs are blessed with the flexibility of both -er and -or suffixes?

neglect -- neglecter; neglector
reject -- rejecter; rejector
detect -- detecter; detector
concoct -- concocter; concoctor

And, what is with the only -ct verb that I could find that only allows the -er suffix. Seriously, what gives? Maybe there are other words that fit in this category, but I can't think of any.

subtract -- subtracter

That's all I really wanted to say.