Limiting people to one job is a death stroke. I once signed up to be a farmer, and I was given the tools and seeds, and that was it. I wasn't allowed to mine (or risk banishment) or anything outside of my job. I know no one here will go to that extreme, but people will lose interest when they have to just mine all day. You could incorporate the miners, shop-keepers, farmers, etc. into a big group that's separate from the army, but are more like minutemen...no....what's the word? A watch, but the people are conscripted...That way they can fight, do their job part time, and still have fun.
I'm thinking about doing something like that as well. My job (hopefully) will be an archer, but I won't stand around waiting for battle. I think a side-job as a merchant will be just as fun, and rewarding.