We consider spectrum management in Cognitive Radio (CR) networks. We model primary users (PUs) activity on a channel and consider finite number of secondary users (SUs). We study the trade-off between two conflicting objectives of minimizing overall data queue lengths for the system and maintaining the quality of service to users. We use Markov Decision Processes (MDPs) to model this and derive an easy to implement scheduling policy. Using simulations, we demonstrate and compare the performance of our policy.