This book contains many results and methods developed during the last twenty years that are not included in any other books. The book covers problems with multiple criteria, and in addition to discrete time problems it considers two types of continuous time problems. The book presents techniques based on the convex analysis and analyses of the geometrical properties of strategic measures and limiting distributions. The book is suitable for researchers and graduate students in the areas of electrical engineering, operations research, management, and applied probability. It assumes an undergraduate course in probability.