Well, I guess the town I live in and helped build is within orcish borders and we have to leave. I hadn't even SEEN any orcs until yesterday when one came by, yelled at me for being on his land, and stabbed me with a spear.
I think you do have enough people to fight the Orcs. But then again, it would require being a bit sneaky with the time the Warclaim is at and to make sure people don't play their Orc alts for the battle.